Deploying Machine Learning Models with Docker

A notebook that produces a 0.94 F1 score has delivered no value yet. Value arrives when something else can call the model, reliably, at a latency someone specified. Containers are how you get from one to the other.

A pickled model on its own is a liability. Reloading it a year later against a different library version fails in creative ways. Ship an artefact bundle: the weights, the preprocessing pipeline, the exact library versions, the input schema, and the training data hash. Prefer a portable format such as ONNX when the framework allows it, because it decouples serving from the training stack entirely.

Validation at the boundary prevents the majority of production incidents, because most of them are malformed input rather than bad maths.

```python
from fastapi import FastAPI
from pydantic import BaseModel, Field
import onnxruntime as ort
import numpy as np

app = FastAPI() session = ort.InferenceSession('model.onnx', providers=['CPUExecutionProvider'])

class Request(BaseModel): tenure_months: int = Field(ge=0, le=600) monthly_charge: float = Field(gt=0) contract_type: str

@app.get('/health') def health(): return {'status': 'ok'}

@app.post('/predict') def predict(req: Request): features = build_features(req).astype(np.float32) score = session.run(None, {'input': features})[0][0] return {'churn_probability': float(score), 'model_version': MODEL_VERSION} ```

Always return the model version in the response. When someone disputes a prediction from three weeks ago, that field is the difference between an investigation and a shrug.

Multi-stage builds and CPU-only runtimes usually cut image size by an order of magnitude.

```dockerfile
FROM python:3.12-slim AS build
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

FROM python:3.12-slim COPY --from=build /install /usr/local WORKDIR /app COPY model.onnx app/ ./ RUN useradd -m app && chown -R app /app USER app EXPOSE 8000 CMD ['uvicorn', 'main:app', '--host', '0.0.0.0', '--port', '8000'] ```

Pin base image digests, not floating tags. A silent minor version bump in a numeric library is a genuinely unpleasant way to lose a day.

• Load the model once, at startup, never per request. This is the single most common performance bug in ML services.
• Batch when latency allows. Grouping requests over a short window multiplies throughput, especially on GPU.
• Set explicit resource limits and thread counts. Frameworks that grab every available core will fight each other under a container quota.
• Separate liveness from readiness. The container is alive well before the model finishes loading.

Infrastructure metrics tell you the service is up. They cannot tell you the model has quietly become wrong. Log input distributions, prediction distributions, and whatever ground truth eventually arrives, then alert on drift. A model degrading slowly with no alarm is worse than one that crashes loudly.