Production

Docker: Base image (e.g. NVIDIA PyTorch), model weights (or mount at runtime), Python deps, inference engine (vLLM/SGLang/TensorRT-LLM), system packages (ffmpeg for audio/video). Pin all versions for reproducible builds. Use --no-cache when debugging build issues.

Keep images lean — multi-GB images slow deployment and cold start. New-model day-zero often needs nightly builds; plan for stable releases shortly after. Consider model-as-volume: mount weights from object storage or shared filesystem rather than baking into image — faster iteration on models without rebuilding.

Layers: Base → deps → app. Order matters: change deps less often than app code. Container layer is ephemeral; changes at runtime are lost on restart. Use env vars for config; secrets via orchestration (K8s secrets, Vault).