One of the lesser arguments against certified reproducible results is that research will cost too much to reproduce. The past 30 years of biotechnology is a testament that this logic is flawed. Non-reproducible cargo cult science is expensive. Pharm research has wasted an estimated trillion dollars in the past decade. If you are into math and science, one trillion is an impressive number when we are talking dollar bills. Something must be done or the good that is in the life sciences will be put on a shelf until some other generation of people take a different path. Until then we will continue to lose our jobs while the bullshitters live out their reckless scientific careers.
What I mean by reckless scientific careers can be seen in the Amgen study. When 47 out of 51 oncology papers turn out to be non-reproducible you have a problem. People use this information. Scientists must rely on their peers but clearly they must do so only through the scientific method. Most scientists do not have the resources to try and reproduce 51 papers. They thus have to go on faith. Faith has failed. It's time to get science police on the job, armed with the scientific method, to go out and do the work that the average scientist cannot afford to do.
Andrew Carnegie made a few bucks in his day. He wanted the world to buy his steel because he felt that it was an amazing material that could be used in mass quantities as the world became more modern. Bridges, railroads, skyscrapers and the list goes on. He saw the power of steel as well as the financial power that was possible. In order to convince the people that steel was the latest greatest building material he put on a show. It was thought that elephants would instinctively not walk across a bridge that they felt would not support them. In a publicity stunt he had an elephant walk across a steel bridge. The bridge was of course strong enough to hold the elephant. People became believers with stunts like this and Canegie became the richest man on the planet.
Science is also powerful. It is used these days in the place of elephants. We can calculate what it takes to build long lasting bridges and tall buildings. But the kind of science we use in biotechnology and pharma R&D is not quite so strong. Currently that kind of science can be used to extract large sums of money from Cargo Cult Leaders and some of their followers, but it has not led to any revolutionary changes in health care. What could we do with the real power of science if it were forced to be strong, like the science behind the building of a bridge or a skyscraper?
Andrew Carnegie saw the potential in steel. Somewhere there must be a businessman/woman who sees the potential in keeping the life sciences alive with a reenforcing steel like force that keeps the companies and the papers from crumbling to the ground the moment someone tries to use them to advance our field. It will take vision and showmanship. Someone must begin to demonstrate the strength to fight against the nay-sayers. Who will our Carnegie be? Has he/she been born yet? There are many reasons why any reproducibility initiative will fail in the world of science. We need to start thinking about why it will succeed. I think that good reprocuble science will make us a whole lot more money the the Cargo Cult Science.
2 comments:
I'm a huge fan of the blog but you have this reproducibility initiative wrong.
As long as the incentives are not directed towards reproducibility, it won't be happen. The Amgen study was great but we didn't hear about until years after the original work and the authors had left the company and felt free to publish.
And of course we still don't know which studies or which labs were involved due to confidentiality agreements.
For academics funding is limiting and precious. If you tell an academic "here is $10,000 and you can spend it on reproducing your own work or you can spend it on a new project which will bring you more money and more publications", which will they choose?
If you ask academics, all of them will insist to you that their work is reproducible so why would any of them consider trying to reproduce it?
tldr: You don't get into Science and Nature by reproducing published research.
Thanks for shharing this
Post a Comment