Science Friday · Science Friday and WNYC Studios

Why so many studies can’t be replicated

April 11, 2026·18 min·1 clip

Researchers found they could only replicate half of thousands of social science papers analyzed in a major DARPA-funded project.

Science asks how trust gets earned. The host frames the episode around a basic question: how can people tell whether a scientific result is trustworthy? Then the guest gets into the work behind the paper. Some published research depends on coding choices that reviewers and readers may never see, even when the paper itself sounds polished and confident. The examples are concrete. A researcher might study minimum wage and unemployment, or test whether a policy changed deforestation in the Amazonian rainforest. None of that analysis happens by magic. Satellite data, policy changes, and control variables have to be gathered, cleaned, matched, and merged before anyone can make a claim. That is where errors can slip in. The guest points to duplicate records, gaps between what a paper says and what the code actually does, and routine analysis steps that may never get checked during publication. The host keeps the conversation grounded. When coding comes up, the follow-up asks what that means for a layperson, turning a technical workflow into something easier to picture. Excel matters here. So do R, Python, and the other tools researchers use to run regressions and turn messy evidence into a result. Peer review, the guest argues, does not automatically cover all of that. A research assistant may write code, a paper may go to experts, and a journal may accept it. Still, no one may have opened the underlying data or tested the code. The point is not that researchers are usually dishonest. It is that science can lean too much on trust when the work could be checked. Openness changes the odds. The guest contrasts older habits, when other people's code was almost impossible to inspect, with newer students who can learn from far more shared code. Replication becomes practical scrutiny, not just a formal ideal. The episode treats code review as part of scientific rigor. When studies shape claims about policy, economics, or the environment, the hidden steps behind the result matter too. Even the guest's self-deprecating laugh about bad old code lands as a serious point: science gets stronger when its working parts are visible.

As heard by us

A clear Science Friday segment on how unchecked data and code can weaken published research.

Science Friday frames the replication problem as a question of how science earns trust after the first finding. Ira Flatow starts with a clean reminder: research is not just the first search, but the follow-up work that checks whether a result holds.

Read the full review in PlayNext →

Why you'd press play

The peer reviewers never looked at the data. That explains quite a lot.

Read the full recommendation in PlayNext →

Listen to the show on

Apple Podcasts Spotify

Play clips from this episode More from Science Friday