Build Reproducible Python Environments with Apptainer (Singularity) — For Researchers Tired of “It Works on My Machine”

Your Python script runs flawlessly on your laptop. Your colleague runs it on theirs and gets import errors. You submit it to the HPC cluster and it crashes with missing CUDA libraries. You spend three hours debugging version conflicts instead of doing research.

This is dependency hell, and it’s the silent killer of reproducible science.

The good news? A single portable file—called a SIF (Singularity Image Format)—can package your entire conda environment, all pip packages, and system-level dependencies. Once built, it runs identically on your machine, your colleague’s machine, and a 10,000-GPU cluster.

That’s Apptainer.

What This Is

Apptainer (formerly Singularity) is a containerization platform purpose-built for scientific computing and high-performance computing (HPC). Unlike Docker, Apptainer requires no root privileges to run, making it cluster-safe. It bundles your entire conda environment—Python version, libraries, system dependencies—into one .sif file that behaves identically everywhere it runs.

Why Apptainer over Docker?

• No root privilege required — cluster-safe; Docker demands root access
• GPU-optimized — built for HPC workloads, not DevOps
• One portable file — no registry, no layers, no fuss
• Simpler mental model — a .sif file you copy and run, not a blueprint you pull

Think of it this way: a Docker image is a blueprint you pull from a registry and run as a container. An Apptainer image is a single self-contained file you copy, move, and execute anywhere.

Prerequisites

You’ll need:

• Apptainer (v1.0+) or Singularity (v3.8+) — installation guide
• conda or miniconda installed locally
• A Linux-based system (native Linux, WSL2, or macOS with Lima)
• sudo access (only during image build, not runtime)

Check your setup:

apptainer --version
conda --version

Getting Started: Build Your First Image

Step 1: Export Your Conda Environment

Start with the environment you’ve already built locally:

conda env export > conda.yaml

Open conda.yaml and delete any line starting with prefix: (Apptainer doesn’t need absolute paths). Your file should look like:

name: myenv
channels:
  - conda-forge
  - defaults
dependencies:
  - python=3.10
  - numpy=1.23.0
  - pandas=1.5.0
  - pip
  - pip:
    - scikit-learn==1.2.0

Step 2: Create a Singularity Recipe

Create Singularity.def:

Bootstrap: docker
From: continuumio/miniconda3:latest

%files
    conda.yaml /opt/conda.yaml

%post
    apt-get update && apt-get install -y build-essential
    /opt/conda/bin/conda env create -f /opt/conda.yaml
    conda clean --all -y
    apt-get clean
    rm -rf /var/lib/apt/lists/*
    rm /opt/conda.yaml

%environment
    export PATH=/opt/conda/envs/myenv/bin:$PATH
    export CONDA_DEFAULT_ENV=myenv

%runscript
    exec /opt/conda/envs/myenv/bin/python "$@"

Key sections:

• %files — Copy conda.yaml into the container
• %post — Install packages (runs once during build)
• %environment — Set shell variables at runtime
• %runscript — Default command when you execute the image

Replace myenv with your actual environment name from conda.yaml.

Step 3: Build the Image

sudo apptainer build myenv.sif Singularity.def

This downloads the base image, installs all packages, and writes a myenv.sif file (~2–4 GB). First build takes 5–15 minutes; subsequent rebuilds are faster due to caching.

Step 4: Test It

apptainer run myenv.sif python --version

If you see your Python version, you’re done.

Core Workflow

Run a Script

apptainer run myenv.sif python my_script.py

Interactive Shell

apptainer shell myenv.sif

Type exit to leave.

Access Local Files with --bind

Your container is isolated by default. To access files on your system:

apptainer run --bind /home/user/data:/data myenv.sif python /data/my_script.py

This maps /home/user/data on your host to /data inside the container.

Common patterns:

• --bind /home/user/project:/work — mount your project folder
• --bind /tmp:/tmp — share temporary files
• --bind /cluster/storage:/data — access cluster storage

GPU Access

On HPC clusters with NVIDIA GPUs:

apptainer run --nv myenv.sif python train_model.py

The --nv flag exposes GPUs and CUDA libraries. Verify it works:

apptainer run --nv myenv.sif python -c "import torch; print(torch.cuda.is_available())"

Share Your Image

Send your colleague three files:

• myenv.sif — the container image
• Singularity.def — the recipe (for transparency)
• conda.yaml — documentation

They run it without installing anything:

apptainer run myenv.sif python their_script.py

Practical Example: Deep Learning Pipeline

Your conda.yaml:

name: medseg
channels:
  - pytorch
  - conda-forge
  - defaults
dependencies:
  - python=3.11
  - pytorch::pytorch::*[build=py311_cuda11*]
  - pytorch::pytorch-cuda=11.8
  - pytorch::torchvision
  - pip
  - pip:
    - monai==1.2.0
    - nibabel==4.0.0

Your Singularity.def:

Bootstrap: docker
From: continuumio/miniconda3:latest

%files
    conda.yaml /opt/conda.yaml

%post
    apt-get update && apt-get install -y build-essential
    /opt/conda/bin/conda env create -f /opt/conda.yaml
    conda clean --all -y
    apt-get clean
    rm -rf /var/lib/apt/lists/*
    rm /opt/conda.yaml

%environment
    export PATH=/opt/conda/envs/medseg/bin:$PATH
    export CONDA_DEFAULT_ENV=medseg

%runscript
    exec /opt/conda/envs/medseg/bin/python "$@"

Build:

sudo apptainer build medseg.sif Singularity.def

Test locally:

apptainer run --bind /home/user/data:/data medseg.sif python train.py --data /data/images --epochs 50

Share with your lab. A colleague on a GPU cluster runs:

apptainer run --nv --bind /cluster/storage:/data medseg.sif python train.py --data /data/images --epochs 50

Same code. Same environment. Same results.

Updating Your Environment

Option 1: Rebuild from Updated Recipe (Recommended)

Edit conda.yaml and add your new package:

nano conda.yaml

Rebuild:

sudo apptainer build myenv.sif Singularity.def

Option 2: Test in a Sandbox First

For interactive experimentation:

sudo apptainer build --sandbox myenv_sandbox Singularity.def
sudo apptainer shell --writable myenv_sandbox

Inside the shell:

conda activate myenv
pip install new-package
exit

Convert back to .sif:

sudo apptainer build myenv.sif myenv_sandbox

Troubleshooting

“apptainer: command not found” — Install Apptainer or check if singularity is available instead.

“conda: command not found” inside container — Verify your %environment section exports the correct environment name:

%environment
    export PATH=/opt/conda/envs/myenv/bin:$PATH

“ModuleNotFoundError” for a package in conda.yaml — Rebuild using a sandbox and manually install to see the error:

sudo apptainer shell --writable myenv_sandbox
conda activate myenv
pip install package-name

Image too large (>5 GB) — Add cleanup to %post:

conda clean --all -y
apt-get clean
rm -rf /var/lib/apt/lists/*

GPU not detected — Use the --nv flag:

apptainer run --nv myenv.sif python -c "import torch; print(torch.cuda.is_available())"

Next Steps

1. Export your current conda environment and build your first .sif.
2. Test locally with --bind to access your data.
3. Share with a colleague and confirm it works unchanged.
4. Version-control your recipe — commit Singularity.def and conda.yaml to Git.
5. Document your workflow — add a README.md showing how to run it.

For research teams, this is transformative: one person builds the environment once, everyone uses the exact same setup forever. No installation. No debugging. Just reproducibility.

What’s your biggest pain point with Python environments right now? Are you already using containers, or is Apptainer new to you? Reply below—I’d love to hear what dependency hell looks like in your workflow.

How do you currently manage Python dependencies across different machines—and what’s stopped you from containerizing before?