In my last post I discussed a paper that I thought was complete bullshit. It was a sciencey narrative that did not stand up to the rigors of further analysis. The logic of the project, on paper, showed a complete lack of appreciation for the complexity of protein folding. The methodology and explanation of the results showed a complete misunderstanding of molecular biology techniques. In the end the project became another failure to be tossed on the steaming heap of bad science.
How does a group of highly educated people get together and do this kind of work? Physical Chemistry scientists take work seriously. They look at a system and determine of work was done. Energy may have been exerted. Heat may have been given off. An exchange of kinetic energy may have taken place but the question remains, was any work done. To look at the above situation you have to look at the system. Many people spent many hours on a hypothesis made by one man. That mans underlings all grunted, "yes yes" and they began adding bells and whistles to a very simple premise. The highest ranking scientist had dreamed up a protein without the benefit of billions of years of evolution. It just came to him and his subordinates had his back. They knew who was signing their paychecks. Fast forward several years. The premise that the molecule would be used to deliver drugs turned out to not be true. Science wins again. No work was done in this system.
So how many people did it take to do this non-work? Energy, money, time, meetings, and all sorts of life science bullshit took place day after day. Each individual involved in this project played a role in trying to validate the narrative set forth by the high ranking scientist. Is that the job of modern day scientists? The emperor was stark raving naked here and no one had the courage to point that out. They did what they do until the project was one day ended. The questions then are what did they do and why?
One of the functions of one of the individuals was to write the paper. Another individual read what was written and made changes. The paper was then sent off for peer review. Several individuals at the JBC then exerted energy and gave off heat in their role. But no work was done. Why did they all do what they did? In the final analysis they were all part of a project that did not pan out. Wouldn't science demand an explanation of the outcome, even though it wasn't the desired outcome? The final analysis by this Cargo Cult member is that the project was flawed from the beginning. The tribesmen who worked on the project were not in the place Feynman wished for us. The ultimate conclusions is that this type of work always fails and we should do something about it.
In Cargo Cults however, we answer to authority. Science is not the authority, it is the whims of the highest ranking scientist. The man/woman who signs the paychecks determine what and why we do what we do. We are not allowed to question that authority. When a paper is written we all put our name on it claiming that we all agree with the guy who wrote it. And the guy who wrote it agrees with the guy who told him to write it. And so on and so on.
Imagine if each individual listed as an author was simply allowed to disagree with any points made in the conclusion section. After the paper is written alternative explanations are allowed and the individuals who offer them are not punished for insubordination. Ah but that would not be an authoritarian system. Imagine the guy in the watch tower with coconuts over his ears saying to the boss, "I don't think I'm doing any good up here."
Thursday, August 30, 2012
Tuesday, August 28, 2012
Barbeque Contests
While sitting around in the early morning I came across a Travel Network show on BBQ contests. The chefs are well known in BBQ circles. Many had won several competitions over and over. When the judges sit down to do their taste tests, they are blinded as to who prepared the food they are eating. Contrast that with the peer review system. Why do we bias our peers with our reputations? Don Polderman had over 500 publications. That is what the peer review system creates. The quantity of Dr. Ps work seemed to have some influence that the quality did not. The quality of work being done by the current group of human beings who are professional scientists in general, as judged by the lack of reproducibility of the work they publish, is not good. As judged by the peers it's the cream of the crop. Apologists and defenders would attack such a statement because science is... Fill in the the rest of that sentence yourself. Science is...
Science is blinding the judges at your BBQ contest to see if they can spot the best chef. Cargo Cult Science is authoritarian. The judges (peers) usually allow only one explanation. This prevents any further judging (unauthorized peer reviews) of the judges (the authoritarian peers). I would like to offer up an example of a Cargo Cult paper that I participated in, briefly why it's Cargo Cult and how the authoritarian system let this take place. It is in fact the norm in science and it is very hard for an Asberger Syndrome sufferer such as myself to understand.
You be the BBQ judge. I won't tell you my name or how I am associated with this paper. I will only offer up an alternative explanation to a few of the Cargo Cult points.
The paper is about the Trp cage and phage dispaly. The two real scientific discoveries were combined to create nonsense. Selecting what phage display library to use for your project is a random process. You select your library on conjecture and sometimes it works out for you. The excuse for its failure is usually that the library lacks the proper structure. Ironically, protein structures and folding are what make us who we are. Proteins are fascinating molecules that have so much complexity in what they can do its hard to imagine how the judges of this work did not question the oversimplification. Why did they think that the Trp cage would be in tact with the randomization of 7 of 20 amino acids? It is simply a sciencey narrative that was never scrutinized. Unlike the harder sciences that employ NMR and computer modeling to analyze the structure of this fascinating little protein, the life science biotechnology folk simply took the concept, drastically changed the amino acid sequence, and made the claim that it was still the Trp cage. Drawing the line from the Trp cage to a phage display scaffold and the finally to drug delivery molecules is a stretch of the imagination that boggles the mind. How did the judges not question the logic?
"Mutations were observed at a frequency dependent on display valency". This is a classic Cargo Cult maneuver. A confusing sentence that looks sciencey but makes no sense. Cargo Cults like to emulate real airports. Cargo Cult Scientists do the same when the speak in hard to understand terms. In this case it's hard to understand that sentence because it is complete nonsense. The mutations were simply the result of having the DNA produced by human beings. Ten separate preparations of DNA were sent from Invitrogen. Only one or two was used to make the library. The rest of the preparations had too many "mutations". This is a detail that a judge/peer would not know to question. Anyone who has ever worked closely in molecular biology will have noticed that not all of their clones have the exact DNA sequences that they designed. You don't need them to because you have clones. You only need one good clone. Discard the rest. In phage display libraries however, you have to take the whole set. The authors came up with this explanation out of A) ignorance of bench level molecular biology and B) a need to be sciencey. The explanation is more sciencey than the more probable one, that is good as they could get it. Lacking the expertise of more skilled phage display teams, and lacking time, money and patience, led this group to banging out their first ever library and it was not perfect. But that lack of perfection was not "dependent of display valency".
The biggest Cargo Cultism about this paper is that somehow the combination of real science (the discovery of the Trp cage) and phage display technology will somehow lead to drug delivery technology. It's an idea. It's a thought. But that is all it is. Once tested it became clear that this was a half baked idea. It was dreamed up by higher ranking scientists who told lower ranking laboratory members to make it all come true. When you work in the industry of drug discovery however, you mustn't let people know that any of the ideas of the superiors are half baked. This paper is a prime example of how far people go to talk and talk and never let anyone look up into the sky to see if the airplanes are coming. Thought experiment: If a molecule binds to a target inside a human body, what will it do? In this case the molecules were suppose to be used as drug deliverers. Besides binding to a target, how would this work?
Those of us down below watching the wheels turn in the minds of our Cargo Cult leaders have little to say over the publications and the patents. We need them in fact to advance our careers. We put this stuff on our resumes and we tell everyone we're smart scientists. But I think I have Asbergers. I have a heightened sense of fairness and this paper is bullshit. Who can you tell? The judges are not blind. They do not like to entertain too many thoughts at once, especially if introduced by the wrong ilk.
Nothing came of the library. It was Cargo Cult. No one ever attempted to reproduce the salient points made in the paper. It was good for JBC because it appeared to be scientific. It was judged by peers impressed with the credentials of a couple of the authors. The journal had a new article that wouldn't cause any trouble. The authors have another publication to add to their total count. But it's bullshit. No one will ever use it again, like much of what gets published.
The paper itself is not important here at the CCS. It is what the paper represents. The authoritarian approach to right and wrong. The indifference of the leaders to what is right and wrong. It accomplished their goals but not the one that was suppose to have scientific merit, that the library had some special properties. The airplanes did not land. The company ultimately closed shop due to bigger Cargo Cult issues. One thing shut this company down however and it was the same thing that got this paper published. Bullshit. There are not blinded judges to weed out bullshit in the Cargo Cults. What the leaders say goes. Luckily for the fans of good BBQ, there is a way of judging right and wrong, good and bad. Someday science will catch up.
Science is blinding the judges at your BBQ contest to see if they can spot the best chef. Cargo Cult Science is authoritarian. The judges (peers) usually allow only one explanation. This prevents any further judging (unauthorized peer reviews) of the judges (the authoritarian peers). I would like to offer up an example of a Cargo Cult paper that I participated in, briefly why it's Cargo Cult and how the authoritarian system let this take place. It is in fact the norm in science and it is very hard for an Asberger Syndrome sufferer such as myself to understand.
You be the BBQ judge. I won't tell you my name or how I am associated with this paper. I will only offer up an alternative explanation to a few of the Cargo Cult points.
The paper is about the Trp cage and phage dispaly. The two real scientific discoveries were combined to create nonsense. Selecting what phage display library to use for your project is a random process. You select your library on conjecture and sometimes it works out for you. The excuse for its failure is usually that the library lacks the proper structure. Ironically, protein structures and folding are what make us who we are. Proteins are fascinating molecules that have so much complexity in what they can do its hard to imagine how the judges of this work did not question the oversimplification. Why did they think that the Trp cage would be in tact with the randomization of 7 of 20 amino acids? It is simply a sciencey narrative that was never scrutinized. Unlike the harder sciences that employ NMR and computer modeling to analyze the structure of this fascinating little protein, the life science biotechnology folk simply took the concept, drastically changed the amino acid sequence, and made the claim that it was still the Trp cage. Drawing the line from the Trp cage to a phage display scaffold and the finally to drug delivery molecules is a stretch of the imagination that boggles the mind. How did the judges not question the logic?
"Mutations were observed at a frequency dependent on display valency". This is a classic Cargo Cult maneuver. A confusing sentence that looks sciencey but makes no sense. Cargo Cults like to emulate real airports. Cargo Cult Scientists do the same when the speak in hard to understand terms. In this case it's hard to understand that sentence because it is complete nonsense. The mutations were simply the result of having the DNA produced by human beings. Ten separate preparations of DNA were sent from Invitrogen. Only one or two was used to make the library. The rest of the preparations had too many "mutations". This is a detail that a judge/peer would not know to question. Anyone who has ever worked closely in molecular biology will have noticed that not all of their clones have the exact DNA sequences that they designed. You don't need them to because you have clones. You only need one good clone. Discard the rest. In phage display libraries however, you have to take the whole set. The authors came up with this explanation out of A) ignorance of bench level molecular biology and B) a need to be sciencey. The explanation is more sciencey than the more probable one, that is good as they could get it. Lacking the expertise of more skilled phage display teams, and lacking time, money and patience, led this group to banging out their first ever library and it was not perfect. But that lack of perfection was not "dependent of display valency".
The biggest Cargo Cultism about this paper is that somehow the combination of real science (the discovery of the Trp cage) and phage display technology will somehow lead to drug delivery technology. It's an idea. It's a thought. But that is all it is. Once tested it became clear that this was a half baked idea. It was dreamed up by higher ranking scientists who told lower ranking laboratory members to make it all come true. When you work in the industry of drug discovery however, you mustn't let people know that any of the ideas of the superiors are half baked. This paper is a prime example of how far people go to talk and talk and never let anyone look up into the sky to see if the airplanes are coming. Thought experiment: If a molecule binds to a target inside a human body, what will it do? In this case the molecules were suppose to be used as drug deliverers. Besides binding to a target, how would this work?
Those of us down below watching the wheels turn in the minds of our Cargo Cult leaders have little to say over the publications and the patents. We need them in fact to advance our careers. We put this stuff on our resumes and we tell everyone we're smart scientists. But I think I have Asbergers. I have a heightened sense of fairness and this paper is bullshit. Who can you tell? The judges are not blind. They do not like to entertain too many thoughts at once, especially if introduced by the wrong ilk.
Nothing came of the library. It was Cargo Cult. No one ever attempted to reproduce the salient points made in the paper. It was good for JBC because it appeared to be scientific. It was judged by peers impressed with the credentials of a couple of the authors. The journal had a new article that wouldn't cause any trouble. The authors have another publication to add to their total count. But it's bullshit. No one will ever use it again, like much of what gets published.
The paper itself is not important here at the CCS. It is what the paper represents. The authoritarian approach to right and wrong. The indifference of the leaders to what is right and wrong. It accomplished their goals but not the one that was suppose to have scientific merit, that the library had some special properties. The airplanes did not land. The company ultimately closed shop due to bigger Cargo Cult issues. One thing shut this company down however and it was the same thing that got this paper published. Bullshit. There are not blinded judges to weed out bullshit in the Cargo Cults. What the leaders say goes. Luckily for the fans of good BBQ, there is a way of judging right and wrong, good and bad. Someday science will catch up.
Wednesday, August 15, 2012
The Loss of Tribesmen
Roche announced the closing of its New Jersey R&D hub taking away 1000 jobs. Amgen is going to shut down their Longmont plant where Epogen was produced. They employ around 400 people. Dendreon is cutting around 600. These are just a few recent stories of the Cargo Cults shedding their tribesmen. I worked with a lot of truly science minded people in the industry. Mostly however, I worked with Cargo Cult thinkers. When you have a majority of the latter, you have a sick industry with little need to keep the people around.
Long ago I floated the concept of biotechnology companies being fires lit along the runway of a Cargo Cult Airport. Since I began this blog we've gone through massive layoffs and massive company failures. All but one of the companies that I worked for have called it quits. They all shut down because of bad science fueled by an absolute requirement for specific results. I left the airport a couple years ago but I continue to watch the fires burning out. I continue to have conversations with scientists still in the business and I continue to sense the Cargo Cult mentality. As the industry continues to toss out it's tribesmen and tribeswomen, one has to wonder if the cumulative loss in expertise will affect its future productivity. If we are dealing with a true Cargo Cult, we should expect no loss. The guy in the watch tower with the coconut shell head gear really isn't doing any good. He won't be missed.
The tribesmen may very well be going through an accelerated rate of job loss. New jobs are not being created and old ones are disappearing. Our leaders are the tribe members who are the best politicians thus they know to shield the publics eyes from the real numbers. Even the tribesmen who lose their jobs know its best to not draw attention to the possibility that our industry has disproportional job losses compared to other career paths. The numbers would most likely paint a picture of an unsuccessful group of human beings. This group was given billions of dollars, a wealth of scientific knowledge and they failed.
Cargo Cult leaders have a way of measuring the success of their lives and it's not based on successfully getting the airplanes to land. The cargo they seek has always been money. They measure success on how much money they earn before their fire burns out. IPOs are allegedly making a comeback which was a major factor in our dismal history. Many of the people that I've met base their success on maintaining employment within their cargo cult. Machiavellian tactics trump the scientific method. How many of these people are now sitting on that pile of discarded 2012 biotech/pharma scientists? With accelerated job loss comes a smaller population of leaders.
The true measure success is the ability to maintain the kind of integrity described to us by Richard Feynman in Cargo Cult Science.
So I have just one wish for you--the good luck to be somewhere where you are free to maintain the kind of integrity I have described, and where you do not feel forced by a need to maintain your position in the organization, or financial support, or so on, to lose your integrity. May you have that freedom.
A biotech company can still succeed when the measurement of success is an IPO or a few guys at the top getting rich. If we switch that measurement to reproducible work that is of some use, we will be doing science. We can still read the narratives and be impressed by the minds who dreamed up the possibilities. But then we have to get to work to find out if what they are saying is true. Some of the discarded scientists need to speak up and help us root out the Cargo Cultisms. The Reproducibility Initiative is a great idea and one that will be met with great opposition. If it survives we may have a new sheriff in town. Those who have lost their job, those who hate their current job and those who have tired of losing their investment money will have a new ally. I truly believe that opposition to this initiative is an example of Cargo Cult Science. Your science is suppose to work. It's suppose to be reproducible. Technology is the practical application of science. If your science is bullshit, your technology will not work. The planes will not come.
Subscribe to:
Posts (Atom)