Irreproducibility

Irreproducibility refers to the failure to obtain the same results when a study is reanalyzed or when its methods are repeated by independent researchers. In practice, discussions of irreproducibility are tied to the broader reproducibility crisis, an observation that a substantial portion of published findings across disciplines have not been reproduced in subsequent work. Different communities distinguish between reproducibility (recreating results using the same data and code) and replication (obtaining consistent findings with new data using the same methods). Irreproducibility can involve data analyses, reported statistics, or the substantive conclusions of a study.

Causes of irreproducibility are diverse. Statistical practices such as p-hacking, optional stopping, low statistical power, and failure to correct for multiple comparisons can inflate false positives. Research design and reporting issues, including flexible analysis plans, HARKing (hypothesizing after results are known), selective reporting, and publication bias, also contribute. Practical problems such as incomplete methodological detail, lack of access to data or code, poor documentation, and unstandardized protocols hinder replication. Differences in samples, materials, or environmental conditions can affect outcomes, and in some cases irreproducibility reflects genuine context dependence rather than error.

Consequences include erosion of scientific trust, wasted resources, and difficulties in evidence synthesis. Conversely, recognizing irreproducibility can spur methodological reforms and improvements in research practices. Responses include preregistration and registered reports, open sharing of data and code, enhanced documentation, and dedicated replication efforts. Structural changes—such as incentives that reward robust methods, replication studies, and transparent reporting—are also advocated to improve reliability across fields.