Code is Science is back: the journal code policy tracker now covers 24 journals.See the tracker

OpenAI study: scientists say AI speeds up research code, but checking it is the hard part

A new OpenAI study of 1,819 scientists finds research code is where AI saves the most time. It is also where scientists spend the most effort checking the answers.

NewsBy the Code is Science editorsNov 17, 20265 min read
Line illustration of a scientist at a desk with two laptops and two monitors of code blocks, a magnifying glass over one screen and lab flasks nearby
Key takeaways
  • OpenAI studied 580,186 scientific messages from 1,819 academic scientists; 13.8% of the average scientist's messages were about research software and code.
  • Surveyed scientists reported the largest time saving for code, 2.85 times, but the figure is self-reported.
  • Checking or correcting AI output was the most common difficulty, named by 45.4% of respondents.

On 16 November 2026 OpenAI's Economic Research team published Science at the Frontier: How scientists use ChatGPT, a study of 580,186 scientific messages sent by 1,819 academic scientists in its ChatGPT for Academic Researchers pilot, plus a survey of 251 of them. The ANI news agency and Unite.AI both reported the same headline numbers the next day.

For readers of this site, one finding matters most. Writing research software was the activity where scientists said AI saved them the most time. It was also an activity where they kept going back to check and fix the answers. Fast code that nobody has checked is a familiar problem in research.

Most groups will use these assistants through a chat window. Some go further and build a language model into their own tools, for example to label data or draft analysis scripts inside a lab pipeline. That is generative AI integration work. It needs the same tests, review and version history as any other code that produces results.

What did OpenAI measure?

The study looked at ChatGPT and Codex activity between 20 August and 22 September 2026, and at a survey run from 30 September to 7 October. OpenAI says it sorted messages with privacy-preserving classifiers instead of reading the conversations, and that pilot data were not used for model training by default.

The pilot itself is growing. OpenAI says 20,000 researchers in 33 countries have joined since July, and it now aims to reach 200,000, double its original goal.

How much of the use was research code?

Research software and code development made up 13.8% of the average scientist's scientific messages. It was the smallest of four activities that together made up 85%, according to the figures reported by Unite.AI:

ActivityShare of messages
Reading and knowledge acquisition27.7%
Writing and communication26.9%
Scientific analysis16.7%
Research software and code development13.8%

Requests for fully autonomous research, where the AI would run the literature review, ideas and analysis end to end, were 0.4% of messages. 15.3% of scientists tried it at least once. In practice, scientists used the tool for single tasks, not whole projects.

In the survey, scientists estimated that one hour of work with AI would have taken about 1.9 hours without it at the same quality. The estimate ranged from 1.36 times for administration to 2.85 times for research software and code.

Why is checking the main difficulty?

45.4% of surveyed scientists said checking or correcting outputs was their main difficulty in getting value from ChatGPT. The study also measured how often a request was followed by a clarification, a check or a revision. The average was 31.8%, and 33.7% for research software. Tasks that needed more specialist knowledge drew more follow-up, not less.

That matches what we see in code review. A generated function usually runs. The question is whether it computes the right thing on your data, with your units and edge cases. The errors in our list of common errors in research code, such as off-by-one indexing, silent type conversion and wrong default arguments, are the kind of mistake a fluent assistant can make without any warning.

What should a research group do with AI-written code?

Treat it like code from a new colleague: useful, and checked before it touches a result.

  1. Commit it like any other code. Generated code goes into version control with a clear message, so you can see later which lines came from where. Our guide to version control for researchers covers the basics.
  2. Write the test before you trust the output. A small test with a known answer catches most wrong-but-plausible code. See testing research code.
  3. Have a person review it. A second reader with domain knowledge costs little and catches what tests miss. Our review checklist works for generated code too.
  4. Say what you used. If AI tools wrote part of the analysis code, note it in the methods or the README, and check what your target journal's author guidelines say about AI tools.

What are the limits of the study?

The productivity figures are estimates by scientists who chose to join an AI-access pilot, not results of a controlled experiment. OpenAI's paper itself says the findings do not establish causal effects. The survey also points to limits that AI does not fix: 51.8% of respondents named limited time or staff as a constraint, and 43.8% named funding. For research code, the time saved writing it may simply move to checking it.

Questions about the OpenAI study

How many scientists were in the OpenAI study?

The usage analysis covered 580,186 scientific messages from 1,819 academic scientists in the ChatGPT for Academic Researchers pilot. 251 of them answered a follow-up survey.

How much did scientists use ChatGPT for code?

Research software and code development made up 13.8% of the average scientist's scientific messages, behind reading, writing and scientific analysis.

Does the study prove AI makes research faster?

No. The time savings are self-reported by pilot participants, and the paper says its findings do not establish causal effects.

Should AI-generated research code be reviewed?

Yes. Checking or correcting outputs was the most common difficulty in the study, named by 45.4% of surveyed scientists. Test and review generated code like any other code before it affects a result.

Keep reading. More from the desk.