Reproducible

The number one argument in favor of working towards reproducible science is the "The Drunk Under the Streetlamp". "The Drunk Under the Streetlamp"looks for his keys where the lighting is good. His friends threw his keys in the ditch far from the light so he wouldn't find them and drive home drunk. His mind is not clear so instead of hailing a cab he looks for the keys under a streetlamp. Non-reproducible narratives work in this manner. Non-reproducible narratives will never get the job done. Why do we allow them to predominate the journals? Because they are easier to work with, just as it is easier to see under a streetlamp. Just ask Diederik Stapel! But they are not science. Science works. Science is powerful. When you work on non-reproducible narratives, "the best story", you are not conducting science. We know the current system of publication does not involve the scientific method. The scientific method is too hard. It would be too expensive to bring about reproducible work. Lets take a look a few arguments in favor of the current system.

The first set of arguments comes from yesterdays Retraction Watch comment section. The article was about creating a reproducibility index (RI).

Rather than rate journals on how often their articles are cited by other researchers, let’s grade them on how well those papers stand the most important test of science: namely, does the work stand up to scrutiny?

Seems reasonable. Hold the journals feet to the fire and let them know that we are going to test the research on which they are putting their stamp of approval.  They judge the scientists without using the scientific method. Why not judge them? Not for spite but as a professional courtesy to other researchers trying to use their journal to do work. The first argument against RI:

Reproducibility would take years to achieve, depending on the discipline. It could be one measure to enforce quality control, but it wouldn’t work accross the board. For example, it would be rare to find any group who would be willing to repeat 3-5 season/year agronomic experiments. - Jaime A. Teixeira da Silva

It would take too long. True, those who are the fastest get the rewards. But how is that working out for us? The drunk under the streetlamp, looking for his keys that he knows he lost in the darkened ditch, because the lighting is better, also feels he has a limited amount of time.

Discounting the huge resources necessary to establish a ranking system for which bits of data matter, and which conclusions do not, the whole issue of reproducibility is a hot potato that no publisher’s legal division will want to touch. I foresee litigation in which an author sues a journal for defamation because they down-ranked a paper based on inability to reproduce, when actually the experiment was perfectly sound and the “reproducers” were just not paying attention to experimental details. - Paul Brookes

I will first discount the "huge resources" because we already spend huge resources conducting mostly non-reproducible science. The solution to legal issues is a simple waiver. You waive the right to sue based on the (RI) which is not dictated over by the journal. It is an index that rates reproducibility. If you are trying to publish bad science, you should expect a low rating. If you think the (RI) will hurt your career, be very careful in what you put out into the world. People are watching. You can't just sue a journal because no one else can reproduce your work. But let the lawyers work out those details on protecting journals.

...the idea that journals will somehow willingly accept manuscripts that conflict with established results or confirm what is already known is laughable. 

That is the current situation that people are trying to change. Conflicting with "established results" is what sets science apart from other human endeavors. That is how we "self correct". And yes, even scientists have to fight against "established results" that are not true. In time, the truth will prevail. Think of the (RI) as an index that will make the journals less resistant towards challenging things we know, that aren't so.

Of course the biggest road block here is curating such an index because it’d probably require a lot of reading (or a genius to write an accurate text reading algorithm…key word being accurate).

Again, it's too hard. Too hard because it would require a lot of reading? Why not read a lot? Why not work towards a world where "a lot of reading" is done and reproducibility of what is read is part of what scientists do? It would actually create more work for scientists. Perhaps reproducibility studies could one day employ the minds that would otherwise go to waste looking for jobs at Pfizer running the HPLC department.

In our current system we have plenty of proverbial drunks looking for their keys under a streetlamp. You can spend all night looking. You can put on your glasses and break out a metal detector, but you are not going to find the keys. They are in the ditch where the lighting is not so good. The amount of people defending this foolish system, this system of taking the easy route, maintaining the status quo, works against future generations of scientists. But in 100 years there will be a different system. The truth has a way of making itself known whenever real science is lurking. I can see it now. Real science is making people think about the problems with reproducibility in the year 2013. The current leadership doesn't want a change but real science is lurking. A change is coming, like it or not. You may laugh at those who are fighting for the change but put yourself in historical context. Change does not come easily. If you think reproducibility is not worth fighting for, you are standing in the way. You are now the Nattering Nabob of Negativity. Get out of the way and let this experiment begin.

Stewart Lyman made his arguments against The Reproducibility Initiative back in 2012. Why won't it work? Money, Ego, Time and Science. His argument on Science begins:

Some types of data are easy to replicate, whereas others are much more difficult to reproduce. Cutting-edge experiments may be harder to replicate than “average” data, because, as one of my profs used to tell me “if it was easy, someone else would have already done it”. If you’ve worked in a lab for any length of time, you know that when trying to reproduce someone else’s results, the devil is in the details. 

Science can be hard. The devil is in the details. That is why the Reproducibility Initiative and Index, PloS ONE, and all of the other efforts to clean up this mess, should be considered scientific pursuits. If it was easy, someone else would have already done it. The lack of money and time are the weapons of egotistical scientists who always seem to have more money. They always seem to get results faster than others. They are winning but real science is lurking in the background. Real science is hard. It takes time and plenty of money. But it's worth it. Bad science is not worth fighting for.