Python Production Engineering

Build a CRUD API, scaffold it properly, add logging, containerize, test, secure, observe, and deploy — one layer at a time.

11 Modules 45 Lessons Intermediate Free & Open Source
Start Course View on GitHub
PythonFastAPIstructlogOpenTelemetryPrometheusVectorKafkaElasticsearchVaultDockerKubernetesTrivy

What You'll Build

Every module adds a layer to a real production system. By the end, you have a complete, deployable application.

Production API

CRUD API with FastAPI, Pydantic validation, async database operations, structured logging, and containerized from day one

Full Observability

Structured JSON logging, distributed tracing with Jaeger, Prometheus metrics, and a complete log pipeline with Vector, Kafka, and Elasticsearch

Secure & Deployable

DevSecOps with Trivy, Bandit, Gitleaks. Secret management with Vault. Deploy to Marathon, Swarm, or Kubernetes

Course Curriculum

11 modules · 45 lessons · Each starts with a real problem you'll solve

45 lessons available
1
Build a CRUD API

I need a working Python API — from zero to CRUD

5 lessons
1.1
Why Python for Production Services Python beyond scripts — async services, type safety, and the ecosystem for production
1.2
FastAPI — Async-First HTTP Framework Build your first async API with automatic OpenAPI docs
1.3
Pydantic — Request & Response Validation Type-safe data models that validate at runtime
1.4
RESTful Route Design with FastAPI Router Organize routes with APIRouter, path parameters, and dependency injection
1.5
Error Handling & Response Models Consistent error responses, exception handlers, and HTTP status codes
2.1
Project Structure with pyproject.toml Modern Python packaging — no more setup.py or requirements.txt
2.2
Environment-Based Config with pydantic-settings Type-safe configuration from environment variables with validation
2.3
Async Database Operations SQLAlchemy async sessions, MongoDB Motor, connection pooling
2.4
API Versioning Strategies URL-based, header-based versioning and backward compatibility
3.1
Why Structured Logging Matters grep vs query — why JSON logs change everything in production
3.2
structlog — JSON Logging with Context Configure structlog with processors, renderers, and context binding
3.3
Request-Scoped Logging with Correlation IDs Trace a request across log lines with middleware-injected correlation IDs
3.4
Dual Output — Stdout and File Logging Stdout for containers, RotatingFileHandler for Marathon and host-mount scenarios
4.1
Docker — Containerize from Day One Multi-stage Dockerfile, docker-compose dev workflow, no virtualenv needed
4.2
Docker Compose — Development and Staging Compose profiles for dev, staging — the simplest deployment
5.1
pytest — Fixtures, Conftest, and Async Testing Set up pytest with fixtures, conftest.py, and async test support
5.2
Unit Tests for Business Logic Test validators, transformers, and service layer in isolation
5.3
Integration Tests for API Endpoints Test full request-response cycles with httpx AsyncClient
5.4
Test Coverage and Quality Gates Enforce coverage thresholds in CI — fail the build below 80%
5.5
Pre-Commit Hooks — Shift Left Gitleaks, detect-secrets, Bandit, Ruff — catch issues before they're committed
5.6
GitHub Actions CI Pipeline Test, lint, SAST on every push — automated quality gates
6.1
Container Security — Trivy and Hadolint Scan Docker images for CVEs, lint Dockerfiles for best practices
6.2
Dependency Audit — SBOM and Dependency Track CycloneDX SBOM generation, pip-audit, and Dependency Track dashboard
6.3
Scheduled Security Scans Daily vulnerability scans, license compliance, and alerting
7.1
OpenTelemetry — Traces, Spans, and Context Understand traces, spans, and how context propagates across services
7.2
Auto-Instrumentation for FastAPI Zero-code tracing for HTTP, database, and external calls
7.3
Jaeger — Visualizing Distributed Traces Deploy Jaeger, send traces, and diagnose latency bottlenecks
7.4
Prometheus Metrics — RED Method Rate, Errors, Duration — instrument your API with Prometheus client
8.1
Vector Agent — Lightweight Log Collection Collect logs from files with Vector — no heavyweight agents
8.2
Kafka — Durable Log Transport Buffer logs in Kafka for reliability and fan-out to multiple sinks
8.3
Vector Aggregator — Transform and Route Parse, enrich, and route logs from Kafka to storage backends
8.4
Elasticsearch + Kibana — Search and Visualize Index logs in Elasticsearch, build Kibana dashboards
8.5
End-to-End Pipeline Wire it all together: App → File → Vector → Kafka → ES → Kibana
9.1
HashiCorp Vault — Centralized Secret Management Set up Vault, store secrets, and integrate with your Python app
9.2
Vault Auth Methods — Token vs AppRole Token auth for dev, AppRole for production — when and why
9.3
Secret Workflow — Local to Production .env for local, Vault for staging/prod — a practical workflow
10.1
Marathon (Mesos) Deployment Marathon app definition, health checks, host-mounted log paths
10.2
Docker Swarm Stack Deploy Docker stack, global services, Swarm secrets, rolling updates
10.3
Kubernetes Deployment Deployment, ConfigMap, Secret, HPA, Ingress — full K8s manifests
10.4
Vector Deployment Patterns Sidecar vs DaemonSet vs host-agent — choose the right log collection pattern
11.1
Health Checks — Liveness, Readiness, Startup Implement health endpoints that orchestrators actually use
11.2
Graceful Shutdown and Connection Draining Handle SIGTERM, drain connections, and avoid dropped requests
11.3
Log Rotation and Disk Management RotatingFileHandler config, disk alerts, and cleanup strategies
11.4
Dependency Updates and Vulnerability Patching Automated update workflow — Dependabot, pip-audit, and safe upgrades
11.5
Production Runbook Common failure scenarios, diagnosis steps, and resolution playbook

Prerequisites

Required

  • Basic Python knowledge (functions, classes, modules)
  • Command line comfort (terminal, shell basics)
  • Docker installed (docker and docker compose)
  • Git basics (clone, commit, push)

Helpful

  • REST API concepts (HTTP methods, status codes)
  • Basic understanding of JSON
  • Any prior experience with logging or monitoring
  • Kubernetes or orchestrator exposure

Your Instructor

Jinna Balu

Jinna Balu

Lead SRE · Platform Engineering · Open Source

Building production systems, observability pipelines, and DevSecOps workflows. Every lesson comes from real production experience.

Ready to Build Production Systems?

Start with Module 1 and build your way to a fully observable, secure, deployable Python application.

Start Course — It's Free