Why Kubernetes over Docker Compose?
When building locally, Docker Compose is fantastic for spinning up a few containers. However, in a production environment, you are dealing with distributed systems problems: servers die, traffic spikes unpredictably, and microservices often boot up in the wrong order.
Kubernetes (K8s) isn’t just a container runner; it is an orchestrator with a control plane that constantly monitors the system. If the inference-api pod crashes due to an out-of-memory error, K8s automatically spins up a replacement. Furthermore, it provides internal DNS routing and load balancing out of the box.
Solving the Distributed Startup Race Condition
One of the most common issues in microservice architectures is the “race condition” during startup. If the FastAPI inference pod boots up before the PostgreSQL database is ready, or before the XGBoost model has finished training, the API will crash.
To solve this, the architecture utilizes a specific Kubernetes object: the Job. Unlike a Deployment (which tries to keep a service running forever), a Job executes a containerized task until it successfully completes, and then terminates.
The init-model Job is responsible for training the baseline model and registering it to MLflow.
# k8s/init-model.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: fraud-init-model
spec:
template:
spec:
containers:
- name: init-model
image: fraud-base:latest
imagePullPolicy: Never
command: ["python", "src/ml/train_in_docker.py"]
# Injects all environment variables dynamically
envFrom:
- configMapRef:
name: fraud-config
restartPolicy: Never
backoffLimit: 2 # Fault tolerance: retry up to 2 times if training fails
System Design Note: By deploying this Job first, we guarantee that the MLflow tracking server is populated with an active model before we ever deploy the inference.yaml manifests. It eliminates the fragile “sleep for 30 seconds and hope the database is ready” scripts often found in beginner projects
Decoupling Configuration and Internal DNS
Hardcoding database URLs or API endpoints into application code is a major system design anti-pattern. If the database IP changes, you shouldn’t have to rebuild the Docker image.
Kubernetes solves this using ConfigMaps and its internal DNS resolution. In this project, all environmental configuration is centralized in one K8s manifest.
# k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: fraud-config
data:
# Notice the use of K8s internal DNS (e.g., 'postgres-postgresql', 'mlflow-service')
DATABASE_URL: "postgresql://fraud_user:fraud_pass@postgres-postgresql:5432/fraud_db"
REDIS_HOST: "redis-master"
KAFKA_BOOTSTRAP_SERVERS: "kafka:9092"
MLFLOW_TRACKING_URI: "http://mlflow-service:5000"
Because Kubernetes handles internal routing, the Kafka consumer doesn’t need to know the physical IP address of the FastAPI pod. It simply routes traffic to http://inference-service:8000, and K8s load-balances that request to the healthy pods behind that service.
Securely Injecting Secrets
Finally, the Agentic AI requires a Google Gemini API key to function. Storing this in plain text inside a ConfigMap or a GitHub repository is a massive security risk. Instead, the architecture uses Kubernetes Secrets, which are injected directly into the inference-deployment at runtime.
# k8s/inference.yaml (Simplified snippet)
env:
- name: GEMINI_API_KEY
valueFrom:
secretKeyRef:
name: fraud-secrets
key: GEMINI_API_KEY
By isolating configuration and secrets from the compute layer, the application images remain truly stateless, portable, and production ready.