Who Wants It

I believe that future generations will look back on us as Feynman looked back to the middle ages:

During the Middle Ages there were all kinds of crazy ideas, such as that a piece of rhinoceros horn would increase potency. Then a method was discovered for separating the ideas--which was to try one to see if it worked, and if it didn't work, to eliminate it. This method became organized, of course, into science.

The idea of reproducing someone elses experiments is nothing new. What is new is the idea that such a project would create so much dissent. A piece of rhino horn used for erectile disfunction could be put to the test in a Viagra style study these days. It would be put to the test if some proponent of its use had the clout to get the research put in motion. Therein is the pickle we find ourselves in with reproducibility. The people with the clout are the ones whose work needs to be subjected to reproducibility studies.
Now when I talk about the actual Reproducibility Initiative I am using it as an example of one such effort to get the conversation started about what actaully takes place in the lives and careers of any modern day participant in the scientific world. I think any reproducible science, even boring science that doesn't rise to the sexy standards narrative science, is better than the cargo cult science. You can get rich, you can get published, but that is not the primary concern with this conversation. What I am thinking about is a Utopian world where the members of the cargo cult begin to be skeptics. They lose their jobs in the watch towers because, as their leaders informed them, they were not wearing the coconut antennas properly. They begin to question the antennas. They build confidence in their own ability to distinguish bullshit from the pile of clever sciencey narratives that their leaders pile on them. They discover a method for separating the ideas--to try one and see if it worked, and if it didn't work, to eliminate it.

Having started with this concept, that science began not so long ago with the idea of testing popular ideas, I'll address the comment from my last post. (Thanks for the comment btw)

As long as the incentives are not directed towards reproducibility, it won't be happen. The Amgen study was great but we didn't hear about until years after the original work and the authors had left the company and felt free to publish.

And of course we still don't know which studies or which labs were involved due to confidentiality agreements.

I'm not sure what is meant here. The incentives in reproducible science is that it leads to bigger and better things. We see so far because we stand on the shoulders of giants.

For academics funding is limiting and precious. If you tell an academic "here is $10,000 and you can spend it on reproducing your own work or you can spend it on a new project which will bring you more money and more publications", which will they choose?

Who would be so foolish as to give someone this choice? I've never worked with anyone who would volunteer for an outside reproducibility study of their own work. As I said at the end of my last post however, we have to be creative. Step out of the mindset of an academic scientist. Think in terms of a beaurocrat who decides on the NIH budget or an investor who wants to give biotech one more chance. You have a choice: Fund the same old guys who burned you back in the 90s and 2000s, or tap into a new paradigm. Only fund people who agree to have their work double checked. We're all only human. That is what the same old guys don't want you to know. Don't believe them. If they don't want the money, find someone else.

If you ask academics, all of them will insist to you that their work is reproducible so why would any of them consider trying to reproduce it?

Again, the academics are not the consumers (yet). They are the ones in whom we've lost faith. A Reproducibility Initiative must first do business with the consumers of good and bad science. The consumers can continue to lose money like they have in the past or they can try and see what certifiably reproducible work can do for them.

tldr: You don't get into Science and Nature by reproducing published research.

Again, no one is trying to get into Science and Nature here. They are trying to get Science and Nature to get into reproducible science.
Thanks to whomever this commenter was. You are absolutely correct with everything you said. The issues you bring up are the exact issues to attack. The question is how. Can we find a stunt like the Carnegie elephant crossing a steel bridge stunt? We have to be creative. The academic scientist is not the guy who wants a reproducibility group hanging over their research. In time however, the people with the money will come to find investments in reproducible science to be far more lucrative than the investments they made without this safety net. Soon scientists will have no choice, if they want funding, but to conduct the kind of work that will be subjected to reproducibility.

I know I'm only dreaming. But this is all just a thought experiment.