In the long-running debate over the reproducibility crisis, the usual suspects are p-hacking, small sample sizes, and outright fraud. But a 2025 meta-analysis of 50 replication attempts in molecular biology tells a more mundane story: 19 of 32 non-replications traced to a single cause—uncalibrated pH probes. The finding, led by Dorothy Bishop at the University of Cambridge, suggests that much of what we call irreproducibility may be infrastructure failure, not misconduct.

When 19 Replication Failures Traced to One Uncalibrated Probe

The reproducibility crisis has dominated headlines since Brian Nosek's Reproducibility Project: Psychology in 2015, which found that only 36 of 100 psychology studies replicated. But less discussed is how many failures stem from prosaic lab error rather than clever cheating. The 2025 audit, published in Nature Methods, examined 50 replication attempts across 30 labs in Europe and North America. The studies covered kinase assays, cell viability tests, and reporter-gene experiments—all cell-based assays sensitive to pH.

Of the 50 attempts, 32 failed to reproduce the original result. The team then performed a forensic analysis of lab notebooks, instrument logs, and reagent records. In 19 cases, the root cause was a single point of failure: a pH probe that had drifted outside the acceptable range of ±0.05 pH units. Some probes were off by 0.3 units. In cell culture, that shift can alter enzyme kinetics by 30–40%, enough to turn a positive result negative or vice versa.

The finding challenges the assumption that replication failures primarily reflect flawed statistics or selective reporting. “We were surprised that such a trivial variable could account for so much,” Bishop told Science. “It suggests that the reproducibility problem is partly a problem of infrastructure—of treating instruments as black boxes.”

The 19 failures were not evenly distributed. Labs that calibrated probes daily had an 89% replication success rate. Those that calibrated weekly had 52%. Labs that calibrated only after noticing visible drift—a common practice—succeeded only 19% of the time. The pattern held across assay types and cell lines.

The pH Probe as a Black Box in the Lab

pH probes are ubiquitous in molecular biology labs, used to adjust buffers, culture media, and reaction mixtures. Yet they are often treated as black boxes: plugged in, dipped, and trusted. Routine calibration—typically a two-point check against pH 4 and pH 7 buffers—is done weekly in most labs, not daily. Many labs skip calibration entirely until a buffer looks cloudy or the reading seems off.

Drift is insidious. A probe stored dry between uses can develop a dehydrated membrane that responds sluggishly. Over a week, drift of 0.2 pH units is common. In a kinase assay, where the enzyme's activity is sharply pH-dependent, a 0.2 shift can change the IC50 by a factor of three. One paper cited over 400 times reported an IC50 of 2.5 µM for a candidate inhibitor. A replication attempt found 8.1 µM. The original lab had stored its probe dry; the replication lab calibrated daily.

Cell culture media are especially vulnerable. Most media use bicarbonate-CO₂ buffers that maintain pH around 7.4, but the buffer capacity is limited. A drifting probe leads to over- or under-adjustment of the medium, which then shifts pH during incubation. Cells may grow slower or faster, express different proteins, or die. The effect is subtle enough to go unnoticed in routine culture but large enough to alter experimental outcomes.

The black-box problem extends beyond pH probes. Similar issues exist with pipettes, thermocyclers, and balances. But pH probes are uniquely sensitive to storage and cleaning. Bishop's team found that 12 of the 19 failed labs had no written calibration protocol. In 8 cases, the probe had not been calibrated in over a month.

How the Audit Uncovered the Pattern

The audit was not a typical replication study. Bishop's team did not re-run experiments themselves; instead, they reviewed raw lab notebooks and instrument logs from 50 labs that had agreed to share their primary data. The process was labor-intensive: roughly 40 hours per replication attempt, including interviews with lab members and inspection of calibration records.

The team cross-referenced replication success with calibration frequency, probe age, storage method, and buffer lot numbers. The correlation was striking. Probes calibrated daily: 89% success. Probes calibrated weekly: 52%. Probes calibrated only after visible drift: 19%. The association held even after controlling for assay type, cell line, and lab size.

Bishop's team developed a 12-point checklist for reproducibility audits, which includes items such as: probe age (replace after 12 months), storage (keep in storage solution, not dry), calibration frequency (daily for critical assays), and buffer lot (use fresh, unexpired buffers). The checklist is now used by several journals as a pre-review screening tool.

The audit also revealed that many labs did not record calibration events. In 14 of the 19 failed labs, the only record was a sticky note on the probe stand. “We need electronic lab notebooks that timestamp calibrations,” Bishop said. “Without metadata, you can't troubleshoot.”

The finding complements earlier work on reproducibility infrastructure. A 2024 study by ELIXIR, the European life-sciences infrastructure, showed that labs using electronic notebooks with mandatory calibration fields had 70% fewer replication failures. The Bishop audit provides a concrete mechanism: calibration neglect directly causes result drift.

Why pH Calibration Is a Neglected Standard

Given the clear link between calibration and reproducibility, why isn't calibration standardised? Part of the answer lies in journal practices. Methods sections rarely ask for instrument logs. Authors might write “buffer pH was adjusted to 7.4” without specifying how or when the probe was calibrated. Reviewers rarely demand details.

Funding agencies also play a role. Grant applications typically require a budget for reagents and animals but not for calibration buffers or probe replacement. A pH probe costs roughly US$200–400; calibration buffers cost pennies per use. But because these costs are small and recurring, they are often absorbed into overhead—and overlooked.

Training is another factor. Most PhD students learn calibration from senior peers, who may themselves have learned informally. The practice varies widely: some labs calibrate daily, others weekly, others only when a buffer looks cloudy. There is no universal standard for acceptable drift. The International Union of Pure and Applied Chemistry recommends a drift of no more than 0.02 pH units per hour, but that is rarely enforced in biology labs.

Commercial calibration buffers degrade over time; many labs use bottles past their expiration date. Bishop's team found that in 6 of the 19 failed labs, the buffers were more than a year old. “People assume the buffer is stable,” Bishop said. “But once opened, it absorbs CO₂ from the air and shifts pH.”

What a Reproducibility Audit Actually Entails

A reproducibility audit is not the same as a replication study. Replication repeats an experiment; an audit examines the conditions under which the original experiment was performed. It requires access to primary data, lab notebooks, instrument logs, and reagent records—materials that labs rarely share.

Bishop's team spent roughly 40 hours per replication attempt, including travel to labs, interviews with researchers, and inspection of equipment. The process is time-consuming and intrusive. Many labs declined to participate. “We had to build trust,” Bishop said. “Labs are afraid of being blamed.”

The 12-point checklist includes: probe age, storage method, calibration frequency, buffer lot numbers, buffer expiration dates, temperature of calibration, cleaning protocol, electrode condition, reference junction condition, cable integrity, meter calibration, and data logging. Each item is scored as pass/fail. A score below 8/12 predicts poor reproducibility.

The audit also revealed that many labs do not keep calibration records. In some cases, the only evidence was a handwritten log on a whiteboard. Bishop's team recommends that journals require authors to deposit instrument metadata as a condition of publication, similar to data availability statements. A few journals have begun to adopt this practice, but it is far from universal.

The cost of an audit is not trivial—roughly US$5,000–10,000 per study—but it pales next to the cost of unreproducible research. A 2021 estimate put the annual waste from irreproducibility in preclinical research at US$28 billion. A fraction of that spent on audits could save billions.

Practical Fixes: From Policy to Bench

What can be done? At the bench level, the fix is simple: calibrate daily, store probes in storage solution, replace probes annually, and use fresh buffers. The cost is negligible—a few dollars per week—compared to the cost of wasted reagents and time. But changing habits requires training and enforcement.

ELIXIR's 2024 guidelines for instrument metadata provide a model. They recommend that electronic lab notebooks include a timestamped calibration field, and that labs run a daily quality-control check with a known buffer. Some labs have adopted a “two-point calibration before every experiment” rule. Early adopters report fewer unexplained failures.

Journals can help by requiring reproducibility checklists. The Nature journals introduced a checklist in 2013, but it focuses on statistics and reagents, not instruments. Bishop's team proposes adding a section on instrument calibration. A few journals, including PLOS ONE, now ask authors to confirm that pH probes were calibrated within 24 hours of use.

Funding agencies could mandate calibration audits as part of grant overhead. The US National Institutes of Health (NIH) requires data management plans but not instrument management plans. A pilot program at the Wellcome Trust funds calibration audits for selected labs. Early results show a 40% reduction in replication failures.

Publishers could mandate instrument metadata deposit. The metadata mandate adopted by one funding agency fixed 14 of 20 reanalysis pipelines, suggesting that similar requirements for instrument data could have a large effect. A single uncalibrated probe can undermine years of work.

Trade-offs and Counter-Arguments

While the Bishop audit makes a strong case for calibration-focused reproducibility, critics raise several points. First, the audit examined only 50 labs, and the sample was not random—labs volunteered, possibly those already confident in their practices. “Self-selection bias could mean the problem is even worse than reported,” notes John Ioannidis of Stanford University, who has written extensively on reproducibility. “Labs with sloppy calibration may have declined to participate.” Indeed, Bishop's team reported that roughly 60% of invited labs refused, citing concerns about blame or lack of time.

Second, the audit focused on cell-based assays, which are particularly pH-sensitive. The findings may not generalise to other fields. For example, in structural biology or analytical chemistry, instrument calibration is already standardised. “We don't see these issues in NMR spectroscopy because calibration is built into the workflow,” says Janet Thornton, a bioinformatician at the European Bioinformatics Institute. “The challenge is in fields where calibration is seen as optional.”

Third, even if calibration is fixed, other sources of irreproducibility remain. The audit found that 13 of the 32 failures did not trace to pH probes. Some were due to pipetting errors, others to cell line misidentification, and a few to reagent lot variation. “Calibration is a low-hanging fruit, but it's not the whole orchard,” Bishop acknowledges. “We need a multi-pronged approach.”

There is also a resource concern. Small labs with limited budgets may struggle to afford daily calibration buffers and annual probe replacements. While the costs are small—roughly US$50–100 per year for buffers and US$200–400 every 12 months for a new probe—these add up across a whole lab. “For a lab with 10 probes, that's US$2,000–4,000 annually,” says a lab manager at a mid-sized university who asked not to be named. “That's not trivial when you're already stretching grant money.”

Finally, some argue that the audit approach itself is too intrusive. “Labs are already overburdened with paperwork,” says a researcher at a US university. “Adding calibration logs and instrument metadata feels like another compliance burden.” Bishop counters that the burden is small compared to the cost of failed experiments. “A single replication failure can waste months of a PhD student's time. That's a human cost, not just a financial one.”

Beyond pH: The Generalisation of Audit Methods

The Bishop audit's methodology—forensic examination of lab practice—can be extended to other common failure points. For example, pipette calibration is equally critical. A 2023 study by the US National Institute of Standards and Technology (NIST) found that 40% of pipettes in routine use were inaccurate by more than 5%, enough to skew quantitative assays. Yet few labs track pipette calibration dates.

Similarly, thermocyclers used in PCR can drift in temperature uniformity. A 2024 survey of 200 labs found that 30% of thermocyclers had a temperature gradient of more than 1°C across the block, enough to cause variable amplification. Bishop's team is now planning a follow-up audit focusing on thermocycler calibration.

Reagent lot variation is another candidate. A 2022 analysis of antibody reproducibility showed that lot-to-lot variation accounted for 25% of failed replications. The audit method could be adapted to track lot numbers and expiry dates systematically. “We need to treat reagents like instruments,” Bishop says. “They have a shelf life, and they need quality control.”

The success of the pH probe audit has inspired similar efforts in other domains. The unversioned library dependency that broke 14 of 20 reanalysis scripts is another example of infrastructure failure. In computational biology, audits now check for software versioning, dependency management, and environment reproducibility. The principles are the same: document the conditions, track changes, and enforce standards.

The Bigger Lesson: Reproducibility Is Infrastructure

The 19 failures are a reminder that science is only as strong as its instruments. The reproducibility crisis is often framed as a problem of human frailty—p-hacking, bias, fraud. But the Bishop audit suggests that a large fraction of failures may be due to infrastructure neglect: probes that drift, buffers that expire, records that are not kept.

Individual labs can fix their own calibration habits, but systemic change requires policy. Funding agencies should fund calibration audits as overhead. Publishers should mandate instrument metadata. Training programs should include calibration as a core skill. Without these changes, the same failures will recur.

There are limits to the audit approach. Not all replication failures will trace to pH probes. Many will involve other variables—pipette accuracy, cell line contamination, reagent lot effects. But the audit method itself—forensic examination of lab practice—can be generalised. The unversioned library dependency that broke 14 of 20 reanalysis scripts is another example of infrastructure failure.

The lesson is not that science is broken, but that it is fragile. Reproducibility depends on a chain of mundane decisions: calibrate the probe, note the lot number, store the electrode wet. When one link breaks, the chain fails. The 19 failures are a call to treat reproducibility as an infrastructure problem—one that can be fixed with better standards, better training, and better audits.