I have been taking stock of my 50 years since I left Wichita. How I have existed fills me with horror, for I failed at everything. Spelling, arithmetic, writing, swimming, tennis, golf, dancing, singing, acting, wife, mistress, whore, friend... even cooking. And I do not excuse myself with the usual escape of not trying. I tried with all my heart. - Louis Brooks
I haven't failed. I've just found 1000 ways that don't work. - Thomas Edison
Failing is subjective. The problem with the average judgement of success and failure is that the outcomes are valued more than the effort (the trying). If there was formal instruction on how to conduct research, there would need to be a semester or two on failure. What is it and how do we deal with it? When is it good and when is it bad? How can we write about failure in ways that will be helpful to others?
How can we truly judge an effort (the trying) without the bias that comes from knowing the outcome?
When I was a Boy Scout we had a meeting with about 30 of us boys. The Scout Master told us we were going to have a contest to see who could hold their breath the longest. I took in a deep breath and held my breath for as long as I possibly could, about 50 seconds. The winner clocked in around 4 minutes. This was, of course, a lesson in Boy Scouts never telling lies. When it was over the grown ups had a good laugh. I forget what the joke was, but the Scout Master made his point. No one can hold their breath for that long. In order to achieve the outcome we wanted (winning a prize for being #1) we would have to cheat. The only question the Scout Master needed answered was how long we would cheat.
In my class on coping with perceived research failure, I would start with Feynmans Cargo Cult speech.
But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school--we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation.
We can't hope that the kids will catch on to the ways of scientific investigation. We can face this dilemna head on. In order to teach lessons on failure we must define what it is. How do we distinguish between a failure, such as using the wrong reagents, versus failing to get the results we expect? How can we teach a student how to look at failure the way Thomas Edison did?
Having worked with Cargo Cults so often I can design several laboratory experiments to test the students in a similar manner to my Boy Scout leader. For example, phage display using New England Biolab kits. See how the students treat the repeat sequences and/or contiminants. Test how the students think about the results. There are papers that have been published that tell us what are the most probable explanations for certain sequences are. Will the students find those papers in their attempts to get to the truth? Will the students make the same mistakes that others have made in their attempts to move the research along to more interesting avenues of creating a narrative.
We've learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature's phenomena will agree or they'll disagree with your theory.
And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven't tried to be very careful in this kind of work. And it's this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.
