IN BRIEF
The 9 October 2026 Nature Medicine study screened 3,967 clinical prediction papers citing TRIPOD or TRIPOD+AI.
Only 482 (12.2%) shared code; researchers also assessed 380 repositories for documentation and reproducibility features.
The sample was restricted to articles retrievable as open access through PubMed Central, so it is not a measure of all clinical prediction studies.
WHY IT MATTERS
Models used to estimate diagnosis or prognosis can influence care. Independent review requires more than a reported performance metric; analysts need to inspect how data were processed and results calculated.
A repository link is not the same as an executable, reproducible analysis. Missing software versions, dependencies or instructions can make a published result difficult to check.
WHAT IS NOW POSSIBLE
Before: incomplete evidence on how often clinical prediction models share usable analytical code. Now: a defined 3,967-paper review measures code availability and repository documentation gaps.
WHAT MADE IT POSSIBLE
Researchers used an LLM-assisted pipeline to find code availability statements and assess repositories across 14 predefined features.
The pipeline was compared with 500 manually annotated papers. The validation supports use at scale, though automatic classification can still be wrong.
THE STORY
How easily can other researchers check the calculations behind clinical diagnostic or prognostic models? An international team reviewed 3,967 published articles and found shared analytical code in only 482, or 12.2%. Their study, published in Nature Medicine on 9 October 2026, documents a transparency gap in a field where independent reproducibility is important for trust and clinical deployment.
The sample was not a random representation of every medical model. It included studies citing TRIPOD or TRIPOD+AI reporting guidance that developed, updated or evaluated multivariable prediction models. Full texts also had to be retrievable through the PubMed Central Open Access API. This selection may favour research already attentive to good reporting and could overestimate sharing across the wider literature.
Code sharing was counted when papers made their analytical code available in a repository or supplemental material. The team then examined 380 accessible repositories for characteristics that help another scientist rerun an analysis. Many lacked clear software dependencies, specific package versions or automated tests. Finding a link therefore does not mean that another laboratory can reproduce the calculations successfully.
A second development was methodological: a language-model pipeline helped screen papers and extract code links at scale. Against 500 papers manually reviewed by researchers, it achieved a weighted F1 score of 0.97 for article eligibility and 92.3% accuracy for repository link retrieval. Those figures show useful screening performance rather than perfect classification, so reported percentages should be understood as estimates of the pattern.
Sharing improved over the years within this cohort: 14.5% of papers from 2024 shared code, compared with 15.8% in 2025. Papers citing TRIPOD+AI had higher sharing prevalence than those citing TRIPOD, but this association does not prove that the newer guideline caused the change. The findings are intended to inform the upcoming TRIPOD-Code guidance. They do not prove that studies without shared code are clinically wrong; they show why independent checks are harder.
WHAT WE KNOW
482 of 3,967 selected papers shared code, representing 12.2%. [1]
Accessible repositories varied considerably in documentation, dependencies, licensing and testing.
The automated review was evaluated on 500 manually annotated articles.
WHAT WE DON'T KNOW
The percentage cannot be generalized to all healthcare AI or clinical prediction research.
Failure to share code does not prove that a diagnostic or prognostic model is inaccurate.
Repository availability alone does not establish that analyses execute successfully end-to-end.
WHAT HAPPENS NEXT
Develop clearer expectations for dependencies, software versions, licensing and execution instructions.
Evaluate whether future TRIPOD-Code guidance improves practical reproducibility.
Extend the assessment to other research populations and directly test executable workflows.
SOURCE TRACE
482 of 3,967 selected papers shared code (12.2%).
The researchers assessed 380 accessible repositories.
The automated screen was checked on 500 manually evaluated papers, weighted F1 0.97.
The TRIPOD-citing open-access cohort is not the entire clinical prediction literature.
SOURCES
- AI-assisted scoping review of code sharing in clinical prediction model researchOpen source
- AI-assisted scoping review of code sharing in clinical prediction model researchOpen source
CURRENT EVIDENCE
PEER-REVIEWED
PEER-REVIEWED OBSERVATIONAL META-RESEARCH / NOT A CLINICAL INTERVENTION
WHAT WOULD CHANGE THIS?
- Broader human validation might adjust automatic classification estimates.
- Different study populations would clarify how general the findings are.
- Direct execution tests could distinguish available from runnable code.