MozFest 2017: why is research code not open?

This is a historical note. It describes an event run by the volunteers behind the original Code is Science site in 2017, rewritten by the current editors for the relaunched archive.

In autumn 2017 the project ran an interactive session at MozFest, the Mozilla Festival, held that year in London at the end of October. MozFest has long been a gathering for people who care about an open internet, open source and open science. It was a natural place to ask a question that had been bothering a lot of researchers and developers: if code produces the results in a paper, why does almost nobody see it, let alone check it?

What was the problem?

By 2017 a large share of published research depended on software. Some of it was well known packages. Much more of it was custom scripts written by the researchers themselves to clean data, run models and make figures. Those scripts were usually not published, not licensed for reuse and not looked at by peer reviewers.

That matters for a plain reason. Code is part of the method. A bug in a script is a mistake in the method, and a mistake nobody can see is a mistake nobody can catch. The short version of the pitch was this: research code needs to be open source to be useful, and it needs to be open to be reviewed.

Who was it for?

The session was pitched broadly on purpose. The organizers wanted the people who write research code and the people who build open source software in the same room. You were a good fit if any of these applied:

• You write open source code, or care about it.
• You write software for research. Maybe you call yourself a research software engineer, maybe you just call yourself the person in the lab who knows Python.
• You wanted to understand how academic publishing works and where code fits into it.
• You already believed research software should be open and wanted to do something about it.

No prior experience with peer review or with any particular language was needed.

How did the session work?

The format was short and hands on. It ran for about 60 minutes.

1. A brief intro. A few slides to frame the issue: how code fits into the research record, and why it so often falls out of it.
2. Post-it notes. Everyone wrote ideas on sticky notes, one idea per note, no filtering.
3. Group discussion. The notes went up on the wall, got clustered into themes and discussed as a room.

Post-its are a deliberately low tech choice. They let quiet people contribute as much as loud ones, they make it easy to spot when five people said the same thing in different words, and they leave a physical record that can be photographed and written up afterwards.

Which questions did it ask?

Three prompts structured the discussion:

• Why is scientific code so often not peer reviewed, and why is that accepted? The second half of the question mattered. The goal was not only to list obstacles but to understand why the research community treats unreviewed code as normal.
• What would fix it? Concrete changes: tools, policies, training, incentives.
• Who needs to be influenced? Journals, funders, institutions, reviewers, supervisors and researchers themselves all have a part. Knowing which lever to pull first is half of any campaign.

What was supposed to come out of it?

The plan was to write up the outputs and publish them on the site, which happened in a follow up post on the session notes. Participants who wanted to stay involved were invited to join the project.

One concrete goal was to build a list of journals and their policies on code: which ones require it to be shared, which require an open source licence and which say nothing at all. A list like that is useful in two directions. Authors can see where their code will be expected, and advocates can see which journals still have work to do. The broader hope was to nudge research culture a little further toward openness.

People who could not attend were encouraged to send their thoughts by email or add them to the public MozFest discussion online.

Why start with a conversation, not a manifesto?

It would have been easy to arrive with a list of demands for journals. The organizers chose to listen first. Researchers who keep code private usually have reasons, and some of those reasons are good ones: data that cannot be shared, employers who own the rights, credentials buried in config files. A campaign that ignores those realities tends to lecture people who are already overloaded.

Starting with the question "why not?" produced something more useful than a slogan. It produced a map of obstacles, each of which suggests a specific fix: training for missing skills, guidance for licensing questions, templates for documentation, and policy changes where incentives are the real problem.

Why revisit it now?

Because the questions have aged well. A lot has improved since 2017. Archiving code with a DOI is routine, more journals ask for code availability statements, and dedicated venues for reviewing research software have grown. But the core tension the session explored is still here: the incentives in research reward papers, and code is still too often treated as a private means to that end.

The relaunched Code is Science picks up the same thread with practical guides on licensing, archiving, reproducible environments and reviewing code. The room in London is a good reminder of where it started: a group of researchers and developers with sticky notes, trying to name the problem honestly before trying to solve it.