Publication bias is a problem in all fields of research. The results of a paper should actually receive zero weight in the evaluation of its quality, otherwise there is the motivation to cherry-pick the data that give the most impressive result. The measure of quality should be the way the results were obtained – size of sample, experimental procedure, endpoints used. Ideally the reviewers of a paper should not see its results at all, only the description of the experiment.
The claims presented here are pretty outlandish. Particularly relevant to "Hematology and Oncology" we now know that mice housed under different conditions with different microflora can have vastly different outcomes in any model, not just cancer. To suggest academic incompetence or outright unethical behavior is offensive, and is a particularly narrow view of why experiments are difficult to reproduce.This commenter seems to be defending non-reproducible results. Scrolling down we get a comment on this comment.
Reproducibility is the crucial part of scientific method. If indeed "mice housed under different conditions with different microflora can have vastly different outcomes in any model" (quot. from one of the comments above) then I wonder if mice presents a useful model-study that can be used outside a particular lab and on which further investigations can be built upon outside that one lab.
Perhaps the commenter critical of the Amgen Study does not have a firm grasp on what makes for good science. The natural urge to simply defend could come from the human condition of favoring positive outcomes. As the first commenter points out, results shouldn't matter. That is a very interesting concept that not everyone will understand.
But all of this leads me to the latest TED Talk from Ben Goldacre. He too brings up the Amgen Study. He talks about this idea of positive results and our tendency, as humans, to favor them even when negative results are the accurate results. Hiding them leads to trouble, scientifically speaking.
Again, Feynman CCS
One example of the principle is this: If you've made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out. If we only publish results of a certain kind, we can make the argument look good. We must publish both kinds of results. I say that's also important in giving certain types of government advice. Supposing a senator asked you for advice about whether drilling a hole should be done in his state; and you decide it would be better in some other state. If you don't publish such a result, it seems to me you're not giving scientific advice. You're being used. If your answer happens to come out in the direction the government or the politicians like, they can use it as an argument in their favor; if it comes out the other way, they don't publish it at all. That's not giving scientific advice.