Making Jupyter notebooks reproducible

Notebooks are one of the best tools research has for exploring data and explaining a result. Code, output and prose sit side by side, and a reader can follow the reasoning step by step. They are also one of the easiest places for an analysis to stop being reproducible without anyone noticing.

The problems are well known and the fixes are not complicated. This guide covers hidden state, the restart-and-run-all habit, pinning the kernel and environment, moving logic into modules, keeping outputs out of version control, and sharing notebooks so others can run them.

Why do notebooks break reproducibility so easily?

Because a notebook remembers everything you ran, in the order you ran it, not the order it appears on the page. The kernel keeps variables alive between cells, so what you see on screen can depend on cells you edited, deleted or ran out of order an hour ago.

This is called hidden state. A few common forms:

• A variable was defined in a cell you later deleted. The notebook still works for you, and fails for everyone else.
• Cell 12 was run before cell 8. The execution counters show it, but nobody reads them.
• A filter was applied twice because a cell was run twice.
• A package was installed or imported interactively and is not listed anywhere.

None of this is a flaw in your science. It is how the tool works. The goal is to make sure the notebook, run from top to bottom in a fresh kernel, produces the results you report.

What is restart-and-run-all, and when should I use it?

Restart-and-run-all means restarting the kernel, which clears all state, and then running every cell in order. If the notebook finishes without errors and gives the same results, it does not depend on hidden state. Do it before every commit and before sharing.

In JupyterLab and the classic notebook interface, the option is in the Kernel menu. You can also run a notebook from the command line, which is what you want for automation:

jupyter nbconvert --to notebook --execute analysis.ipynb --output analysis.ipynb

If a full run takes too long to do often, that is a signal. Either the notebook should read precomputed intermediate results produced by a script, or it should accept a small-data option for quick checks. Tools such as papermill let you run a notebook with parameters, for example a sample size, which helps with both.

A useful rule: a notebook is not finished until it has passed restart-and-run-all at least once with the exact environment you will share.

How do I pin the kernel and environment?

Create a dedicated environment for each project, register it as a named kernel, and record its contents in a file committed with the notebook. Otherwise the notebook runs against whatever happens to be installed on the reader's machine.

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt ipykernel
python -m ipykernel install --user --name myproject --display-name "Python (myproject)"

The notebook file stores the kernel name in its metadata, so a reader who follows the README gets the right one. For R notebooks, the IRkernel package plays the same role, with renv recording package versions.

Pin versions in the environment file, at least for the packages that do the work. A file that just lists pandas will install a different version next year. Exporting exact versions with pip freeze, using a lock file, or building a container are all reasonable. Our post on reproducible environments compares the options.

It also helps to print key versions in the first cell. A line showing the Python version and the versions of your main libraries makes any later mismatch easy to spot.

Why should logic move out of the notebook into modules?

Because functions in a module can be tested, reused and reviewed, and code spread across notebook cells mostly cannot. The notebook should tell the story. The module should do the work.

A practical split:

Keep in the notebook	Move to a module
Loading final inputs	Data cleaning and transformation functions
Calling analysis functions	Model definitions and statistical routines
Figures and tables for the reader	Anything copied into more than one notebook
Narrative and interpretation	Anything that needs a test

Put the module in a src/ folder and import it in the notebook. While you are developing, IPython's autoreload extension reloads edited modules without restarting the kernel:

%load_ext autoreload
%autoreload 2

from myproject.cleaning import drop_short_series

Once the logic lives in functions, you can write tests for it with pytest or testthat. We explain where to begin in testing research code. The notebook gets shorter and easier to read, and reviewers have a clear place to look for the method.

How do I keep notebook outputs out of version control?

Strip outputs and execution counts before committing, so Git tracks your code and text, not images and changing numbers. A notebook file is JSON, and embedded plots, large tables and counters make diffs unreadable and repositories heavy.

Tools exist for this. nbstripout is a common one. It installs as a Git filter, so outputs are removed from what is committed while staying in your local copy:

pip install nbstripout
nbstripout --install          # run once inside the repository

Some teams prefer another approach: pairing each notebook with a plain text script using Jupytext, and committing the script. Either way the principle is the same. Version control holds the source. Outputs are regenerated by running it.

There is one sensible exception. For the version that accompanies a paper, it can be useful to keep a fully executed copy, or an HTML export, as a record of what the notebook showed. Store it as a release artefact or in the archive, not as the working file. Also check outputs for anything sensitive before sharing, since a printed data frame can expose rows you never meant to publish.

How can others run my notebook without installing anything?

Use a service that builds your environment from the repository and opens the notebook in a browser. Binder is the best known. It reads standard environment files such as requirements.txt or environment.yml from a public repository and launches a temporary Jupyter session.

This is valuable for reviewers and readers who want to check a figure without setting anything up. A few limits to keep in mind:

• Sessions are temporary and resources are modest, so heavy computation and large data do not fit.
• The repository must be public, and data must be downloadable or included.
• Free public services can change. Treat them as a convenience, not an archive.

For the long term, archive the repository, including notebooks and environment files, in a service that issues a persistent identifier. Our guide to archiving code with a DOI explains how. Container images built with Docker are another option when the environment is complex.

What does a reproducible notebook project look like?

A reproducible notebook project is a small set of predictable parts. Use this as a checklist:

1. Notebooks numbered in run order, such as 01_clean.ipynb and 02_model.ipynb.
2. Reusable logic in src/, with tests in tests/.
3. An environment file with pinned versions, and a named kernel.
4. Relative paths only. No C:\Users\me\Desktop.
5. Random seeds set at the top of any notebook that uses randomness.
6. Outputs stripped from Git, with a filter installed.
7. Every notebook passing restart-and-run-all in a clean environment.
8. A README saying which notebook produces which figure.

What should I do this week?

Choose the notebook behind your most important current result and give it a clean run. Restart the kernel and run all cells. Fix whatever breaks, which will often be a missing import or a deleted variable. Then create a project environment, register it as a kernel, and commit the environment file. Install an output-stripping filter in the repository. Finally, move the longest block of logic into a function in a module and write one test for it.

That is an afternoon of work. The Project Jupyter documentation covers each tool in more depth when you need it.