Publishing research software in JOSS: what to expect

Research software is hard to get credit for. A tool can underpin dozens of papers and still not appear on its authors' publication list. The Journal of Open Source Software, usually called JOSS, was set up to address that. It publishes short, citable papers about research software, reviewed by people who look at the code rather than only the prose.

The process is different from most journals, and the differences catch people out. This post explains what JOSS is for, what you submit, how review works and what happens at acceptance. Details of the guidelines change from time to time, so read the current author documentation on the JOSS site before you submit.

What is JOSS for?

JOSS is for open source software that has a clear research application. The paper is a way to make that software citable and to have it reviewed. It is not a venue for reporting new scientific results. If your software produced a finding, publish the finding in a domain journal and cite the JOSS paper for the software.

JOSS is run by volunteers as part of the Open Journals project, and it does not charge authors or readers. Papers receive a DOI and are indexed like other journal articles.

Is my software in scope?

It is likely in scope if it is open source, useful to researchers and the result of real, sustained development. The guidelines set out a few conditions. In general terms, the software should:

• be released under an OSI-approved open source licence (see choosealicense.com or our post on choosing a licence for research code);
• live in a public repository that anyone can browse and where anyone can open an issue;
• have an obvious research use, whether as a library, a command line tool, a model or an analysis package;
• be feature complete enough to use, not a placeholder for planned work;
• represent a substantial scholarly effort, rather than a thin wrapper or a single small script.

Small utilities and one-off analysis scripts are usually out of scope. That is not a judgement on their value. It reflects what the journal is designed to review. If you are unsure, the JOSS documentation explains how editors assess scope, and you can ask before submitting.

What goes into paper.md?

A short Markdown file in your repository, plus a BibTeX file for references. The paper is typically a page or two, not a full article. It begins with a YAML header for metadata:

---
title: 'example-analysis: tools for tidy lab measurements'
tags:
  - Python
  - measurement
authors:
  - name: Alex Doe
    orcid: 0000-0000-0000-0000
    affiliation: 1
affiliations:
  - name: Example University
    index: 1
date: 28 May 2026
bibliography: paper.bib
---

The body is short and has a few expected sections:

1. Summary. What the software does, written for a non-specialist reader.
2. Statement of need. Who it is for, what problem it solves and how it relates to existing tools.
3. Acknowledgements, including funding, if relevant.
4. References, generated from paper.bib.

Do not put API documentation, tutorials or benchmarks in the paper. Those belong in the software's own documentation, and reviewers will look for them there. The statement of need is where papers most often fall short. Be specific about the gap your software fills and name the alternatives honestly.

JOSS provides a way to compile a preview of the paper before you submit, so you can check formatting and references.

How does open review on GitHub work?

Everything happens in public issues in a JOSS review repository on GitHub. The sequence is roughly:

1. Submission. You submit the repository URL and some details through the JOSS website.
2. Pre-review. A pre-review issue opens. An editor checks scope and basic requirements, then looks for reviewers.
3. Review. A review issue opens with a checklist for each reviewer. Reviewers work through it and open issues or pull requests on your repository for anything that needs fixing.
4. Revision. You respond in the review thread and fix problems in your repository. There is no separate response letter.
5. Acceptance. When reviewers are satisfied, the editor asks you to archive the software and then recommends acceptance.

An automated bot handles routine steps such as generating the paper proof and checking references. Reviewers' names are public, as are their comments. The tone is generally constructive: the aim is to improve the software, not to gatekeep it.

Because the review is iterative, it can take some weeks or longer, depending on reviewer availability and how much needs changing. Responding promptly helps more than anything else.

What is on the reviewer checklist?

The checklist covers the software and the paper. Its exact wording is set by JOSS, but the areas are stable:

Area	What reviewers look for
General	Repository is public, licence is present, authors made substantial contributions
Functionality	Installation works as documented, the software does what the paper claims
Documentation	Statement of need, installation instructions, usage examples, API or function documentation
Tests	Automated tests or clear manual steps to verify the software works
Community	Guidelines for contributing, reporting issues and seeking support
Paper	Summary, statement of need, references and quality of writing

You can read the checklist before you submit and treat it as a to-do list. Most review comments trace back to one of these rows. A missing CONTRIBUTING file or a README with no install instructions will come up every time. Our guides on writing a README covers the documentation side, and a basic automated test suite takes care of most of the rest.

What do reviewers usually ask you to change?

Mostly the things that make software usable by someone other than its authors. Requests vary by project, but some come up again and again:

• Installation that only works on the author's machine. Missing dependencies, unpinned versions or steps that assume a particular operating system.
• Examples that do not run. A tutorial that refers to a renamed function or a data file that is not in the repository.
• Thin documentation. Functions with no description of their inputs and outputs, or no overview of what the package is for.
• No tests, or tests nobody can run. Reviewers want to see how you know the software works.
• A vague statement of need. Reviewers will ask how the software differs from tools that already exist.
• Missing community files. No guidance on how to report a bug, ask for help or contribute.

None of these is a reason for rejection on its own. They are the normal content of a JOSS review, and fixing them leaves the software in better shape for every future user, not only the reviewers.

What happens at acceptance?

You make a release, archive it and give JOSS the version number and archive DOI. The editor checks that the archive matches the reviewed software and that its metadata, such as title and authors, matches the paper.

Common choices for the archive are Zenodo and Figshare. If you already use the Zenodo GitHub integration, a tagged release creates the archive automatically. The steps are in our post on archiving code with a DOI.

Once accepted, the paper is published with its own DOI and links to both the repository and the archived release. Readers can cite the paper for the software in general and the archive DOI for a specific version.

How should you prepare before submitting?

• Run the install instructions on a clean machine or container and fix whatever breaks.
• Add a short example that runs in minutes and shows the main feature.
• Make sure tests run with one command and say how in the README.
• Add contribution and support guidelines, even if brief.
• Check that everyone listed as an author contributed to the software.
• Add a CITATION.cff so users cite the software consistently. Our post on citing software in papers has an example.
• Draft paper.md last, once the software is in shape.

What can you do this week?

Open the JOSS reviewer checklist alongside your repository and go through it honestly, as a reviewer would. Note every row you cannot tick. Most projects find a few gaps, usually documentation and tests. Fix those first. Then write a one-paragraph statement of need. If you cannot explain who the software is for in a paragraph, that is the work to do before you submit.