Skip to content

5.4. Software Containers

What is a software container?

A software container is a standardized, self-contained package that bundles an application's code with all its dependencies, including libraries, system tools, and runtime settings.

Think of it as a lightweight, portable "environment-in-a-box." Containers are isolated from one another and the host operating system, but they share the host's kernel. This makes them far more resource-efficient and faster to launch than traditional virtual machines (VMs), which must virtualize an entire operating system.

The primary benefit of containers is consistency. They eliminate the classic "it works on my machine" problem by ensuring that the software runs identically, regardless of where it is deployed.

What is the difference between an image and a container?

This is a fundamental concept that often causes confusion.

  • Image: An image is a static, immutable blueprint or template. It contains the application, its dependencies, and the instructions for what to do when it's run. You create an image by writing a Dockerfile and building it.
  • Container: A container is a live, running instance of an image. You can start, stop, and delete containers. You can run many containers from the same image, each one isolated from the others.

In short: an image is the recipe, and a container is the cake you bake from it.

Why are containers essential for MLOps?

Containers solve several core challenges in building and deploying machine learning systems:

  • Reproducible Environments: They capture the exact state of your environment, from system-level dependencies (like CUDA for GPUs) to specific Python package versions. This guarantees that your model training, evaluation, and inference processes are fully reproducible.
  • Dependency Management: They resolve complex dependency conflicts by isolating the application. No more worrying about whether installing a new library will break an existing project on the same server.
  • Seamless Portability: A containerized application developed on a data scientist's laptop will run without modification on a production server, in the cloud, or on an edge device.
  • Foundation for Orchestration: Containers are the basic unit of deployment for powerful orchestration platforms like Kubernetes, which automate the scaling, management, and deployment of complex, multi-service MLOps workflows.

What is the standard tool for containerization?

Docker is the industry-standard tool for building, managing, and running containers. It provides a straightforward command-line interface (CLI) and a daemon process that handles the heavy lifting of container management. Its core component is the Dockerfile, a simple text file used to define the steps for creating an image.

To get started, install Docker for your operating system. After installation, verify that it's working by opening a terminal and running:

docker --version

While Docker is free for personal and small business use, large enterprises may need a paid subscription. Always check with your IT department about your organization's policies and available resources.

Where should you host container images?

Container images are stored in a container registry, which acts as a centralized repository for your images. The two most common choices are:

  1. Docker Hub: The default public registry for Docker. It's a good place to find official base images for popular software.
  2. GitHub Packages: An excellent choice if your code is already hosted on GitHub, as it keeps your code and its corresponding images in the same place.

To publish an image to GitHub Packages, you must first authenticate, then tag your image with the correct namespace, and finally push it.

# 1. Authenticate with your Personal Access Token (PAT)
export CR_PAT=YOUR_TOKEN
echo $CR_PAT | docker login ghcr.io -u YOUR_USERNAME --password-stdin

# 2. Tag your image
# Format: ghcr.io/OWNER/IMAGE_NAME:TAG
docker tag bikes:latest ghcr.io/fmind/mlops-python-package:latest

# 3. Push the image to the registry
docker push ghcr.io/fmind/mlops-python-package:latest

Your published image will then be available at a URL like ghcr.io/fmind/mlops-python-package.

What does a baseline Dockerfile for an MLOps project look like?

A Dockerfile provides the step-by-step instructions for building your image. The recommended pattern is a multi-stage, non-root build driven by uv: a build stage resolves the locked dependencies, and a slim final stage copies only the ready-to-run virtual environment. This keeps the image small and avoids running as root.

# syntax=docker/dockerfile:1
# Multi-stage, non-root image built with uv (https://docs.astral.sh/uv/guides/integration/docker/).

# 1. BUILD STAGE: resolve and install the locked dependencies into a virtual environment.
FROM python:3.14-slim AS build
ENV UV_COMPILE_BYTECODE=1
ENV UV_LINK_MODE=copy
WORKDIR /app
# Bring in the uv binary from its official image (no separate install step).
# Pinned, not `:latest` — a floating tag is invisible to Dependabot's docker ecosystem.
COPY --from=ghcr.io/astral-sh/uv:0.12.3 /uv /uvx /bin/
# Install dependencies first (cached layer), then the project itself.
RUN --mount=type=cache,target=/root/.cache/uv \
  --mount=type=bind,source=uv.lock,target=uv.lock \
  --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
  uv sync --frozen --no-install-project --no-dev
COPY . /app
RUN --mount=type=cache,target=/root/.cache/uv \
  uv sync --frozen --no-dev --no-editable

# 2. FINAL STAGE: copy only the virtual environment and run as an unprivileged user.
FROM python:3.14-slim
ENV PYTHONUNBUFFERED=1
WORKDIR /app
# Fixed numeric uid/gid: stable file ownership across rebuilds and bind mounts, and
# resolvable by a host that does not share this image's /etc/passwd.
RUN groupadd -r -g 10001 app && useradd -r -u 10001 -g app -m app
USER 10001:10001
COPY --from=build --chown=10001:10001 /app/.venv /app/.venv
ENV PATH="/app/.venv/bin:$PATH"
ENTRYPOINT ["bikes"]
CMD ["--help"]

You can build and run this image with the following commands. Because dependencies are installed from your uv.lock inside the build stage, there is no separate wheel to build beforehand:

# Build the Docker image and tag it as "bikes:latest"
docker build --tag=bikes:latest .

# Run the container, which executes the ENTRYPOINT with the default CMD
docker run --rm bikes:latest

Both commands are wrapped as mise tasks so they are spelled the same way everywhere:

mise run build:image  # docker build --tag=bikes:latest .
mise run docker:run   # docker run --rm bikes:latest

How do you lint your Dockerfile?

A Dockerfile is a build script that runs as root, and like your workflow YAML, it is easy to leave uninspected. hadolint fixes that: it parses the Dockerfile into an AST, applies a rule set of Docker best practices, and runs shellcheck over every RUN command. It is a standalone binary, so it is pinned in mise.toml and wired into the project's static checks:

[tasks."check:dockerfile"]
description = "Lint the container image definition (hadolint)"
run = "hadolint Dockerfile"

Because check:dockerfile is one of the subtasks mise run check depends on, the linter runs on every commit through the pre-commit hook and on every pull request through CI—the same run, no separate configuration.

Two lines in the Dockerfile above exist specifically because a linter or a bot asked for them.

Pin the uv image instead of using :latest

The build stage originally copied the uv binary from ghcr.io/astral-sh/uv:latest. That is convenient and wrong for two reasons.

The first is reproducibility, which you already know: :latest means "whatever was pushed most recently", so two builds of the same commit can embed different tool versions.

The second is subtler and is the reason it got fixed. Dependabot's docker ecosystem updates base images by rewriting a concrete version into a newer one. A floating tag has no version to rewrite, so Dependabot sees nothing to do and stays silent. The image was not "always up to date"—it was outside the update system entirely, and nobody would ever have been told about a uv release or a vulnerability fix. Pinning ghcr.io/astral-sh/uv:0.12.3 makes the version visible: it now arrives as a reviewable pull request like every other dependency.

The same reasoning applies to your FROM python:3.14-slim line: a concrete tag is what makes updates trackable.

Use a numeric USER

The original final stage created a user and switched to it by name:

RUN groupadd -r app && useradd -r -g app app
USER app
COPY --from=build --chown=app:app /app/.venv /app/.venv

Running hadolint on that reports a finding on the USER line (the number below is whichever line it sits on in your file):

Dockerfile:5 DL3066 info: Non-numeric user-id may not be resolvable by host system

The rule is about a boundary the Dockerfile cannot see. Inside the image, app resolves through /etc/passwd to whatever uid useradd happened to allocate. Outside the image, the host kernel only ever deals in numbers, and it does not read your image's /etc/passwd. So a Kubernetes runAsNonRoot check, a securityContext comparison, or a bind-mounted volume's file ownership all operate on a uid that the Dockerfile never stated and that a rebuild could change.

Declaring the number fixes the boundary in place:

RUN groupadd -r -g 10001 app && useradd -r -u 10001 -g app -m app
USER 10001:10001
COPY --from=build --chown=10001:10001 /app/.venv /app/.venv

The uid is now part of the image's contract: stable across rebuilds, meaningful to the host, and unambiguous to an orchestrator. Note that the group is pinned too (-g 10001, and 10001:10001 in USER), because file ownership on a mounted volume depends on both.

10001 is a conventional choice—a high, unprivileged id that will not collide with system accounts on the host.

How can you optimize your container images and workflow?

  • Use Multi-Platform Builds: Use docker buildx to build images that can run on different CPU architectures (e.g., amd64 for cloud servers and arm64 for Apple Silicon Macs). This "build once, run anywhere" approach is highly efficient.
  • Leverage Layer Caching: Docker builds images in layers. Structure your Dockerfile to place steps that change less frequently (like installing system dependencies) before steps that change often (like copying your source code). This allows Docker to reuse cached layers, dramatically speeding up subsequent builds.
  • Minimize Image Size: Smaller images are faster to pull and deploy. After installing packages, clean up cache directories and temporary files. For example, in Debian-based images, add && rm -rf /var/lib/apt/lists/* to your apt-get install command.
  • Lint Your Dockerfile Automatically: Do not run Hadolint by hand when you remember to. Wire it into check:dockerfile so it runs with every other static check, locally and in CI.
  • Pin Every Image Reference: Give every FROM and COPY --from a concrete version tag. :latest breaks reproducibility and hides the dependency from Dependabot.
  • Scan the Image, Not Just the Source: trivy image <tag> inspects the built image's operating-system packages and installed libraries—layers your source-level dependency scanner never sees.
  • Manage GPU Dependencies: For deep learning, your image must include the necessary NVIDIA drivers. Instead of installing them manually, use official base images from NVIDIA, such as nvidia/cuda:12.1.1-cudnn8-runtime-ubuntu22.04.

Additional Resources