Writing · Research Data

How to make your analysis reproducible six months from now

← All writing

Here's the scenario almost every researcher has lived through. A reviewer asks a question about Table 3. You open the folder from last spring, find four scripts with names like analysis_final_v2_REAL.R, run the one that looks right, and get a number that doesn't match the paper. Now you have two problems: the reviewer's question, and the fact that you can't reproduce your own result.

Reproducibility gets talked about as an ethical obligation, which it is, but that framing makes it sound like extra work you do for other people. The practical case is simpler and more selfish. The person most likely to need to rerun your analysis is you, and the version of you who needs it will have forgotten everything. Everything below is about making that future rerun boring.

Keep raw data raw, and never edit it

The single most common way an analysis becomes irreproducible is that someone opened the data file, fixed a few things by hand, and saved over it. Now the cleaning steps exist only in the memory of whoever did them, and only until they forget.

Put the data you received into a folder called data/raw/ and treat it as read only. On Mac or Linux you can literally enforce this with chmod -w. Every correction, recode, filter, and merge happens in a script that reads from data/raw/ and writes to data/processed/. If you find three bad rows, don't delete them in Excel; write the line of code that drops them, with a comment saying why. That comment is the audit trail a reviewer, a coauthor, or a suspicious future you will want.

The test to apply: if data/processed/ and all your figures were deleted right now, could you regenerate them from data/raw/ by running your scripts in order? If yes, your analysis is reproducible in the way that matters. If no, find the manual step and turn it into code.

Pin the software, not just the code

Your code is only half the analysis. The other half is the exact versions of the packages you ran it with, and packages change. A function gets a new default, a statistical routine changes how it handles missing values, a plotting library alters its default color scale. Six months later, the same script on the same data can give you a different answer, and you'll waste a week wondering whether you made an error.

The fix is to record your environment in a file that lives with the project:

Commit the lock file to your repository. It's small, it's text, and it's the difference between "run this code" and "run this code the way I ran it."

Set the random seed, and set it in the script

Anything with randomness in it (bootstrapping, cross-validation splits, multiple imputation, k-means, simulation, train/test splits, any model fit with stochastic optimization) will give slightly different results each run unless you fix the seed. Put set.seed(20260824) in R or np.random.seed(20260824) in Python near the top of the script, not typed into the console once and forgotten.

Two things people get wrong here. First, a seed set in an interactive session doesn't travel with the code, so the script has to contain it. Second, some libraries carry their own random state: scikit-learn estimators take a random_state argument, and if you leave it out, a global seed may not be enough. When a result is important, set the seed explicitly wherever the library gives you the option.

A caveat worth knowing: parallel and GPU computation can still produce tiny numerical differences run to run, because floating-point addition isn't associative and the order of operations changes. Those differences are usually far below anything that matters substantively, but if your conclusion flips when the eighth decimal changes, the problem isn't reproducibility.

Get rid of hard-coded paths

A line like read_csv("C:/Users/samah/Desktop/project/data/survey.csv") breaks on every computer that isn't yours, including your own next laptop. Use paths relative to the project folder instead, and start every script from the project root. In R the here package handles this well (here("data", "raw", "survey.csv")); in Python, pathlib with a path defined relative to the script file does the same job.

The same rule applies to anything else that varies by machine: database credentials, API keys, and cluster-specific directories belong in an environment file or a small config file that is not committed, with a committed example (config.example.yml) showing what fields are needed. Nobody should have to reverse-engineer your file system to run your code, and nobody should find your API key in your public repository.

Notebooks: run them top to bottom before you trust them

Jupyter and R Markdown notebooks are excellent for exploring and terrible at remembering what order you did things in. A notebook can show clean output while its actual state came from cells you ran in a different order, or from a cell you deleted after it defined a variable that everything downstream still uses.

The discipline is simple: before you share a notebook, or before you use its numbers in a paper, restart the kernel and run all cells from the top. If it fails, the notebook was lying to you and it's better to learn that now. For anything that ends up in a manuscript, consider moving the analysis into plain scripts and using the notebook only to present results, which also makes version control far less painful (notebook JSON produces unreadable diffs).

Write the README you'd want to receive

A README is the map. It doesn't need to be long. It needs to answer five questions:

That last one is the one people skip and reviewers most appreciate. A single line like "03-models.R produces output/table3.csv, which is Table 3" saves an hour of guessing. If you want a fuller treatment of folder structure, naming, and documentation, the guide to organizing research data covers it in more detail.

Use version control, even alone

Git has a reputation for being a collaboration tool, and it is, but its bigger value for a solo researcher is that it removes the need for filenames like final_v3_revised_USE_THIS. You get a dated history of every change with a note explaining why, and you can go back to the state of the code on the day you generated a specific result.

You don't need to master it. Four commands (git init, git add, git commit, git log) get you most of the benefit. Commit at the end of every working session with a one-line message about what changed. Add a .gitignore so large data files, credentials, and output don't get committed. If the repository is public, keep data out of it entirely and point to where the data live instead.

Where the data go when you're done

Reproducibility ends at a place other people can reach. Code on your hard drive isn't accessible; code in a repository with a DOI is. Deposit the analysis code and, where you're permitted, the data in a repository that issues persistent identifiers, and cite that deposit in the paper. Most funders now expect this anyway, and journals increasingly ask for a data availability statement that points somewhere real.

Which repository depends on your discipline and how sensitive the data are; the repository guide walks through the choice, including what to do with data about people. If you're writing a data management plan for a proposal, this is the section that describes exactly what you'll deposit and where, so the DMP guide is the companion piece.

A realistic starting point

You don't have to do all of this at once, and retrofitting a finished project is much harder than starting a new one right. If you want the shortest useful version, do these on your next project from day one: raw data untouched in its own folder, a lock file for your environment, a seed set in the script, relative paths, and a README with the run order. That's maybe two hours of setup across the life of a project, and it's the difference between answering a reviewer in an afternoon and rebuilding an analysis from scratch.

If your institution is trying to make this standard practice rather than individual heroics, that's a training and infrastructure question as much as a coding one, and it's the kind of work I help campuses set up. If you'd like to talk it through, reach out.


Samah Alshrief, Ph.D.

AI data scientist and research-computing specialist. I help researchers and universities with research data management, data services, and private AI. Get in touch →