Important overview of the topic-- I am listing several papers that address the issue of reproducibility. For those that are interested, the papers below are free to download from the links provided:
We have unscrupulous people claiming to be scientists. The reproducibility is key, and a scientist worth his salt should question everything including his own results!
I would add to this that we also need a viable option for publishing so called “negative results.” How many of these experiments are repeated ad nauseum, wasting money and time?
One PI that I know refused to accept “negative results” from her trainees. How many of these experiments articles from her lab do you suppose actually had decent science in them?
I just read a commenter the other day who said we were on the verge of curing Alzheimers, until the funding stopped. It's like some people live in a different dimension.
Thank you for this post. We have been hearing about this crisis for years, without much controversy or serious steps towards resolution. The issue at stake has been always research VALUE = OUTCOMES = REPRODUCIBLE RESULTS / COSTS. Now that the new administration and healthcare leadership are taking some serious steps toward addressing the VALUE equation in research and auditing both outcomes and funding, many *scientists* are having an outrage , a meltdown. The problem is WITHIN the scientific community. Lay people understand and support the scrutiny and reform. When you hear pragmatic and objective voices like Dr. Prasad, Dr Ioanidis, Dr. Makary, Dr. Bhattacharya and the alikes -representing many of us , a silent majority- and compare with the loud voices of progressive scientist -activists in their echo chambers where the elite academic and research establishment reign unopposed (see MedPage, NEJM,JAMA, etc.) we realize that we live in separate worlds, likely irreconcilable. That's why the government of the people is compelled to jump to the ring and act as referee, for the betterment of the people they represent and in defense of common sense and science. Welcome Dr Bhattacharya, Dr Makary RFK Jr. Change and reform long due in public health, medicine and scientific research !
The remedy is simple. Every research paper must be required to list the number of times its results have been reproduced according to certain well-defined statistical criteria.
Conclusions from papers that have been reproduced zero times should be described as strictly speculative--pending further research and validation by other independent researchers.
Add: The entity that funds the research requires 'favorable' results, no matter what the honest results demonstrate. Regarding researchers, there's a Darwinian dynamic at work: The company will dismiss an honest researcher who finds unfavorable results, and continue to hire the researcher who gives the sought-after results, irrespective of decent science. In the long run, there are no intellectually honest researchers. This includes statisticians.
Important overview of the topic-- I am listing several papers that address the issue of reproducibility. For those that are interested, the papers below are free to download from the links provided:
Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8): https://pmc.ncbi.nlm.nih.gov/articles/PMC1182327/
Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016) https://www.nature.com/articles/533452a
Ioannidis JP. How to make more published research true. PLoS Med. 2014 Oct 21;11(10). https://pmc.ncbi.nlm.nih.gov/articles/PMC4204808/
There are more, but those papers are behind a publisher's paywall. The articles listed above can be downloaded from the links provided
Your article is why we need SOMEBODY like RFK 2 to shake things up.
We have unscrupulous people claiming to be scientists. The reproducibility is key, and a scientist worth his salt should question everything including his own results!
I would add to this that we also need a viable option for publishing so called “negative results.” How many of these experiments are repeated ad nauseum, wasting money and time?
One PI that I know refused to accept “negative results” from her trainees. How many of these experiments articles from her lab do you suppose actually had decent science in them?
I just read a commenter the other day who said we were on the verge of curing Alzheimers, until the funding stopped. It's like some people live in a different dimension.
Thank you for this post. We have been hearing about this crisis for years, without much controversy or serious steps towards resolution. The issue at stake has been always research VALUE = OUTCOMES = REPRODUCIBLE RESULTS / COSTS. Now that the new administration and healthcare leadership are taking some serious steps toward addressing the VALUE equation in research and auditing both outcomes and funding, many *scientists* are having an outrage , a meltdown. The problem is WITHIN the scientific community. Lay people understand and support the scrutiny and reform. When you hear pragmatic and objective voices like Dr. Prasad, Dr Ioanidis, Dr. Makary, Dr. Bhattacharya and the alikes -representing many of us , a silent majority- and compare with the loud voices of progressive scientist -activists in their echo chambers where the elite academic and research establishment reign unopposed (see MedPage, NEJM,JAMA, etc.) we realize that we live in separate worlds, likely irreconcilable. That's why the government of the people is compelled to jump to the ring and act as referee, for the betterment of the people they represent and in defense of common sense and science. Welcome Dr Bhattacharya, Dr Makary RFK Jr. Change and reform long due in public health, medicine and scientific research !
but universities need life sciences grad students to TA all the pre med students, there's a lot of money in it.
You have nailed the concept on the head!!
The remedy is simple. Every research paper must be required to list the number of times its results have been reproduced according to certain well-defined statistical criteria.
Conclusions from papers that have been reproduced zero times should be described as strictly speculative--pending further research and validation by other independent researchers.
Add: The entity that funds the research requires 'favorable' results, no matter what the honest results demonstrate. Regarding researchers, there's a Darwinian dynamic at work: The company will dismiss an honest researcher who finds unfavorable results, and continue to hire the researcher who gives the sought-after results, irrespective of decent science. In the long run, there are no intellectually honest researchers. This includes statisticians.