Practical Intelligence and Success

The biotechnology and pharmaceutical industry continues to try and figure out new ways to encourage innovation. As patents expire and streams of revenue dry up, they need the next big thing. It's not that they didn't plan on this contingency. They knew these days would come and they shelled out the money to get people working in the laboratory. What happened?

The lack of innovation can be linked to what is ailing all of science. In a recent discussion on scientific journal retractions from "OnTheMedia" I found two comments from people who have interesting and similar concepts.

Paul Charles Leddy

Science is social, and if a bunch of morons decide to do science, they'll create a world of moronic truths. Those of us who don't want to live in a world filled with these "truths" have to be careful not to let the morons say what is truth. I think the past 10 years have shown this over and over. Same goes for journalism, btw.
Sep. 06 2011 12:43 AM

The key point here is that science is conducted by human beings and we tend to form groups of like-minded cohorts. We create the truth. Those with practical intelligence who want to be a success will gear their work towards the truth of the group, not the truth of nature.

p.f.henshaw from way uptown

Brook and Jonah,
You seem to be seeing only the manageable tip of the problem of finding and correcting errors in science. The cases where right and wrong are simple to identify are not the problem. Science never had, and can't have, a way to "purify" its archives, other than the same way nature purifies her complex systems to remove useless branches, by experiment and evolution.

The deep problem of modern science is that "useless branches" of thinking become the basis of social structures and clung to relentlessly. You see it in how the different "silos" of reasoning form around different socially preferred ways to ask the same questions. One dominant paradigm of that kind is "science as computers" with the dazzling display of results conveying the image of powerful insight, but if tested against the subject addressed often represents no insight at all.

Theorists tend not to study nature at all, just data, their theoretical models, and their social status. The naturalists who actually study the complex naturally behaving subjects of such a study are unable to contribute to the process, at all... don't even get brought in for discussion, for the simple reason that nature does NOT behave at all like a computer (!!) and the questions a naturalist would ask upset the social status of someone representing their theory as nature! ;-) See the problem?

So the core of the problem is the social basis of the questions that each science and sub-science organizes itself around, not just our present self-defeating obsession with computers. As a battle between social cells science becomes as much as if some endless TEA Party argument. Your radio piece seemed to assume that scientists were engaged in scientific debate, but you can't do that when people all standing on different platforms.

I love the imagery of useless branches being the basis of social structures being clung to relentlessly. This takes place inside each company but also among all of science such as the case of RNAi. Where was the hint that Merck bailed out on RNAi because after an exhaustive study they came to the conclusion that it will not be useful as a drug? They bailed but they did not send out any dissenting opinion that would upset the social group who still cling to RNAi.

In order to be successful you have to know what the group wants to believe. That is the easy part. The hard part is maneuvering around empirical evidence to the contrary of the groups "truth". For that reason the most successful members of science do not work in laboratories. Those who do lack the practical intelligence that keeps one out of harms way.

What To Do With It

It was not a beautiful paper.

Next came the patents. The news that came out today is that the U.S. Patent and Trademark Office (USPTO) has issued a Notice of Allowance for our patent application.

A primary advantage of this patented peptide library is the ability to rapidly screen and identify novel peptides that exhibit cell specific targeting characteristics for directed delivery of nucleic acid therapeutics," said the Chief Scientific Officer. "Delivery remains a significant challenge in the nucleic acid therapeutic space, and peptides with high affinity and specificity are expected to be a fundamental component to developing delivery approaches to a wide spectrum of tissues and cell types. In addition, the library may also be exploited to screen for peptides that function as specific antagonists, agonists or generally exhibit drug like properties.

The only problem is that the library has no ability to do the things described above. It was a dud. The obedient workers did everything they could but they couldn't get the results that validate the statements of the Chief Scientific Officer quoted here.

The CSO didn't mention that the team who developed and tested the library were sacked. They needed to go out and find someone who could figure out what to do with that library.

There are very few people with the expertise and resources to perform this kind of science.

"What to do with it?" was always the question. There was no conversation about "what to do with it" other than use it to target this cell or that cell. But how? That was the scientific "wealth into the system" that Feynman speaks of. Everything else was what Kurt Vonnegut referred to as "Kit Science" in Cat's Cradle. We had molecular biology kits, phage display kits, and we did some cell culture work. We got our patents and paper. But what should we have done with the library?

A lot of scientists I've worked with believe that there is some clever thing we didn't think of. When we got the orders to test the library against specific cells there was no discussion about the possibility that it wouldn't work. We tested the random thoughts of the leaders, various methods of treating the cells, different buffers and so on. When about 100 random thoughts had been tested they told us to go home. It was a relief.

But it doesn't work. No airplanes land. So
I call these things cargo cult science, because they follow all the
apparent precepts and forms of scientific investigation, but
they're missing something essential, because the planes don't land.

Now it behooves me, of course, to tell you what they're missing.
But it would be just about as difficult to explain to the South Sea
Islanders how they have to arrange things so that they get some
wealth in their system. It is not something simple like telling
them how to improve the shapes of the earphones. But there is one
feature I notice that is generally missing in cargo cult science.
That is the idea that we all hope you have learned in studying
science in school--we never explicitly say what this is, but just
hope that you catch on by all the examples of scientific
investigation.

The CSO makes the claim that "peptides with high affinity and specificity are expected to be a fundamental component to developing delivery approaches to a wide spectrum of tissues and cell types". Why are they expected to contribute to this Holy Grail of RNAi technology? How will it work? Feynman also said:

You can know the name of a bird in all the languages of the world, but when you're finished, you'll know absolutely nothing whatever about the bird... So let's look at the bird and see what it's doing -- that's what counts.

I think of the peptide in this library as the name of a bird. They are looking for that name spelled out in amino acid letters. But even if they find it they will still know nothing about the peptide. They have to see what it's doing. How will they do that? Once again, "what to do with it?". Who will ask the question?

There is nothing wrong with not knowing something. That is why we do research. The problem comes when we want to believe something so much that we accept false answers. This library is just a library. It was not conceived of by someone who had any idea what to do with it. That person, and all of the others are gone now. The CSO quoted above inherited the library and he doesn't know what to do with it. Somehow he's figured out that it will be fundamental to solving the RNAi delivery problem. How will he know when they've found the magic RNAi delivery peptide? I guess they'll know it when they see it.

What Is Really Stifling Innovation

Profit

The Art of Deception

Magic is the only honest profession. The pricing of Seattle Genetics new drug is an example of dishonesty in the biotechnology industry. Clay Siegall, their CEO, is good at the art of deception. He holds a PhD and a large number of patents and publications. You might be lead to believe that he is a scientist. He is actually a businessman. He tells you his company is dedicated to unmet medical needs. That is the diversion. While we are patting them on the back for doing something to help mankind, they are plotting to charge us over 100K to have access to their drug.

The trick is not hard to do. The Invisible Gorilla is an example. In the invisible gorilla experiment there are people in white t-shirts and black t-shirts passing a basketball around. You are told to count the number of times the people in white t-shirts pass the ball to each other. While you are counting passes, someone in a gorilla suit walks through the scene. Most people seem to not see the gorilla because their attention has been diverted by the direction to count passes. You are tricked into not seeing something that is quite obvious once you know it is there.

This is how biotechnology operates, and much of the scientific community as well. Biotech CEOs and principal investigators do not wear white lab coats. They do not work in laboratories. They get noticed. They work very hard to gain a reputation that can be cashed in for money. Once the research grant or investment is in the bank, they go out and hire low paid obedient bachelor degreed servants to do the work. The only requirement is that the results must further the career of the leader.

We at the CCS are always looking for that gorilla walking past the screen. Go to the Seattle Genetics website and count white lab coats.

Did you see the businessmen deciding on the price of the drugs and the size of their bonuses?

Magic is the only honest profession.

Ideas Not Preached from the Pews

The discussion of ideas is necessary for them to catch on. Watching TV or reading a book can be enjoyable but the best shows and books lead to further discussions. You get to participate in the seed that was planted. Watching NOVA or reading a Malcolm Gladwell book may enhance your ability to hold an intelligent conversation. It's up to you to surround yourself with people who also enjoy discussing new ideas. If you enjoy science, you should have friends who have something to share about the latest findings in Science or Nature or any of the other journals. Beyond the mandatory presentations and seminars, you should discuss the latest ideas.

I've had an epiphany on this very subject. I have no friends who enjoy exploring the absurdity of science as it is currently practiced. No one I know has anything negative to say about the profession of science. It seems impolite to be so negative about something that has a positive effect on our society such as science. I believe that only a small percentage of the ideas put forth by scientists ever amount to any greater understanding of our world. And that small percentage is very powerful. The other 95% of our ideas are not going to be useful. They may even be harmful. That 95% of BS deserves greater discussion.

For example, there is a paper out in Nature this week entitled "Broad neutralization coverage of HIV by multiple highly potent antibodies". The title itself seems unscientific. Using words like "highly potent" should be qualified in the conclusions of your paper. But that is the conversation I can't have.

Upon hearing about the HIV paper in Nature, I asked a friend if it would be a good idea to test the authors ability to identify their antibodies by coding each one (A, B, C... Q). Using the techniques reported in Nature, could they retake those measurements to identify each antibody. My friend was shocked that I was nothing but excited about how far science has advanced in HIV research. I said that I would be very excited if I could believe it were true. I was told I was just a skeptic. We did not discuss the paper, just the benefits of believing versus non-belief.

Every Sunday you will find churches full of people listening to a person discuss some aspect of the bible. That one book is discussed in a one way conversation by the preacher to the congregation. Later that conversation is carried on amongst the congregation. The better the sermon the more it will be discussed. Where do scientists gather on a weekly basis to discuss the latest findings? Usually, in the U.S., we gather in rooms where the lowest ranking "scientists" present their data to the highest ranking scientists. The direction of the conversation is taught in graduate school where the wannabe PhD "defends" his/her work to their committee. The preaching among scientists, in other words, is done from the pews, not the pulpit. Subsequent conversations are usually held by disgruntled grad students back at the laboratory, well out of the hearing range of a college professor. It is here where the real scientific conversations take place. Ideas are put forth to squash the dumb ideas. All of the proper controls are thought up. But rarely will those ideas be taken seriously unless they support the ideas of the preacher from the pew.

The mainstream idea to improve our world through science is to increase funding. The idea that I have today is to increase the discussion of scientific work. How to do it? If you read the Huffington Post, you will see that certain articles illicit more comments than others. What if there were a Huffington Post-like science blog where people could discuss work such as the latest HIV/AIDS paper in Nature? As we do with the work of social/economic engineers (our elected leaders) we can discuss the ideas and the merits of the scientific community leaders. Thought leaders such as Andrew Fire and Craig Mello can offer up their thoughts on why RNAi has been such a bust. Throw in a some comments from RNAi biotech CEOs and CSOs and see what the rest of us who are very interested in the subject have to say. We'll get into arguments and have an old fashioned debate, rather than the usual Lead Zepplins found in the journals. Break a paper down to pieces such as the IC50 measurements used in the HIV/AIDS Nature paper. Try and illicit a discussion on the measurements, the use of statistics, and the conclusions that can realistically be drawn. Highlight vested interests and how they effect certain outcomes. Talk about the "sexiness" of the research and how it enhances careers but increases the amount of BS being put forth. Highlight seemingly boring observations and how they can make a big difference.

The preachers from the pews do not seem to inspire. They conspire to keep bad information in the published journals so as to not hurt their careers. They have created a situation where laboratory work is looked down upon. Their greatest sin however, is that they are boring. They do not want the lower ranking members of science to discuss their work. They want us to read about how successful they were and thank them. If we don't believe them or we question them we are nattering nabobs. It flies in the face of what science is all about. If we want to know what really happened in the laboratory, we are going to have to let the people who actually do the work have a say. Let the people who read the papers and try to use them have a say. By increasing the level of talk surrounding science we'll increase the honesty.

The Complexity Issue/Type III IV Errors

An idealist is one who, on noticing that a rose smells better than a cabbage, concludes that it will also make better soup.

H. L. Mencken

Thought experiment: Executive Bill believes that siRNA can be used to knock out TNF alpha and reduce swelling in joints in rats and thus it will do the same in human beings.

Think of Executive Bill as a Martian who has come to the conclusion that humans will prefer rose soup over cabbage soup. He sends his little Martians out to prove him right.

By setting up such a complex hypothesis Bill has failed to acknowledge that the siRNA may not even be the proper molecule that mimics a naturally occurring small piece of RNA that will reduce gene expression. Smell alone may not predict food preference.We fool ourselves all through the hypothesis testing and reach the end point (where Executive Bill demands a full report in his office) with the false assumption that a human being should see a reduction in joint swelling. Executive Bill wants to know one thing, did the human see a reduction in joint swelling? Did the humans prefer rose soup?If the human sees an improvement, Executive Bill is vindicated. If there is no improvement, Executive Bill sends the scientists back into the lab to tweak the system.

In colloquial usage type I error might be called "failing to believe the truth". In biotechnology this is a common error when desired results do not match actual outcomes. Executive Bill will reject the truth (siRNA against TNA alpha has no effect on joint swelling) and send the scientists back to the lab to get the desired results. The Martian leader will send the little Martians back to earth to get them to prefer Rose Soup.

Type II error is "believing the falsehood". This would be the case if the data showed a reduction in joint swelling from the siRNA treatment when the siRNA had nothing to do with the reduction. Executive Bill would eagerly accept this type of error.

The Martian leader would eagerly accept any data that showed humans preferring Rose Soup. Let's say that the problem came from a mislabeling of the soups or fraud was committed by an ambitious little Martian. The desired results were obtained. They were false. They were accepted. Executive Bill and the Martian leader accept what they are being told.

In these two examples, siRNA and Rose Soup, we (the Cargo Cult Scientist) are saying that the non-erroneous outcome would be that siRNA has no effect on TNF alpha or anything in the human body, and that human beings will not eat Rose Soup. Our antagonists, Executive Bill and the Martian leaders' desired outcomes should be proven false. No matter what the outcome, they will only be accepting validation of their hypothesis. The easiest path to success, as defined by Bill and the Martian, is a type II error. Type I errors will be made until a type II can be arranged.

Now let's enter a superior being I will refer to as God. It is the Cargo Cult God and he needs to keep his subjects ignorant for amusement purposes. He delights in the folly of the minds who keep themselves fooled at all times. The Cargo Cult God wants to explain how Executive Bill and the Martian leader reach their status in their respective groups and how they maintain their positions. First, they are bullshitters and thus would use the truth if they could get to it. But getting to the truth would be a random act since they do not know how to conduct research. They begin from ignorance, either type I or type II errors or they are randomly yet unknowingly correct. They then select a desired outcome and draw a line from A) type I or II error or random correct assumption, to B) desired outcome. The desired outcome however cannot be attributed to the path of reasoning that is depicted by the line from A to B.

In 1974, Ian Mitroff and Tom Featheringham argued that "one of the most important determinants of a problem's solution is how that problem has been represented or formulated in the first place".

They defined type III errors as either the error of having solved the wrong problem when one should have solved the right problem or the error of choosing the wrong problem representation when one should have chosen the right problem representation.

In other words, Executive Bill set out to solve the problem of how to make siRNA into an anti-RA drug where he should have tried to solve the problem of determining the possibility of using siRNA as a drug in the first place. Maybe siRNA cannot survive in the human body.

Likewise, the Martian leader wanted to prove that Rose Soup would be preferable to Cabbage Soup. He should have conducted research on the relationship between the human senses of smell and taste.

In 2009, 'Dirty Rotten Strategies' by Ian I. Mitroff and Abraham Silvers was published regarding type III and type IV errors providing many examples of both developing good answers to the wrong questions (III) and deliberately selecting the wrong questions for intensive and skilled investigation (IV). Most of the examples have nothing to do with statistics, many being problems of public policy or business decisions.

The human body is complex. Biotechnology has yet to accept this fact. The study of type III and IV errors has been handed a gift in the form of medical research. How did we get to the point where we are today? 90% inaccuracy in our scientific journals? Publication retractions on the rise for unknown reasons.

The desired (i.e., non-erroneous) outcomes of a test are called true positive meaning "rejecting null hypothesis, when it is false" and true negative meaning "not rejecting null hypothesis, when it is true". What is the case when the desired outcome is not defined as non-erroneous?

Typing on Blue Paper

If you type 100 words a minute with ten years experience typing TPS forms on blue paper for ACME Products, what is your skill?

Typing

If you are the finance officer of a biotechnology company, what is your skill?

Finance

If you conduct research in a life science laboratory, what is your skill?

Research

The question however, is whether or not anyone conducts research in a biotechnology laboratory. Research is not about any particular piece of equipment or assay. An individual who can obtain a degree in science should have the ability to learn how to operate the various pieces of equipment. But biotechnology companies do not conduct research. The board will select the drug target and the antibody/siRNA or whatever is in fashion will be developed to address the target. Most often, they do not match target to technology and they one day will have to close up shop. What is missing is an understanding of research.

Research is an evolution. We begin with ignorance and evolve to understanding. Ignorance has an unpleasant connotation however, one that PhDs do not want associated with their work. Thus, they begin with an ignorance of their own ignorance. How many biotech PhDs have tried to make an antibody against amyloid beta? How many have used phage display to bind to amyloid beta? The concept is that they begin with the knowledge that something that binds to amyloid beta should prevent it from forming plaques in the brain and thus cure Alzheimer's patients. Research did not create that evolution of understanding. This stepwise cure for Alzheimer's has cost billions but has not panned out. There is something we don't know. We are still ignorant about the beginning of our research. Why are the plaques there?

As a result of the inability to admit our own ignorance, we have established a new paradigm for conducting research. We draw out mechanisms, such as we did with RNAi being used as a drug. The siRNA will enter the blood stream, go to its target and begin to reduce the translation of mRNA. Nothing else will happen and the target knock-out will cure or slow down the condition. Early on, it seemed that siRNA was knocking out targets. There was a leap of faith that this could be a drug. Since then we have a large body of evidence (both known and hidden from the light of day) that would lead us to think more about where this research emerged from. Did we begin from ignorance and evolve to understanding? It seems we began from a vague understanding and evolved to ignorance.

What does any of this have to do with identifying a persons skill? Does the secretary type 100 words per minute or does she specialize in typing on blue paper? A researcher in biotechnology is not a person who usually obtains 20 to 30 years of experience conducting research. Rather, they will begin their career as someone who is given the task of knocking out a target to effect a specific condition. The conditions are well documented so we assume the only way forward is to tackle the unknown of dealing with them. You will need molecular biology specialists, cell culture specialists, protein purification specialists, analytical machine operators, and so on. All of the people working in the laboratory are usually non-PhD research associates. Their careers begin with one of the above specialties and they advance their careers only in terms of years of experience. Typing is not enough. They must type on blue paper. For example, I've seen technicians with five years of experience running a Waters HPLC. The laboratory they apply to has Agilent HPLCs. They are out of the running. The skill is required by the researcher. It is not a skill that they must understand, thus they can make the mistake of not identifying what matters most. Do they want someone to effectively use the HPLC for its intended use or do they have an Agilent in their lab that no one knows how to turn on?

The researcher then, should be the person working to put all of the skilled technicians together to move from ignorance to understanding. The most in-demand researchers however are those who have the most patents and publications. The assumption here is that they have reached the level of understanding more often than those with less patents and publications. Science however is a superior way of thinking that takes great pains to distinguish between quantity and quality. Biotechnology is not science, it is a business. It needs experienced obedient workers to turn on the machines and do the paperwork. Research is handled by those who can most often get published and get patents. As a result the amount of useless patents and inaccurate publications have sky rocketed. The companies have had to return time and again to the point of ignorance, the point where research begins. It's not a bad place if you are a scientist. It is a place that fascinates and brings on the evolution of understanding. If you just spent $1.1 billion, such as Merck did on Sirna, it is a bad place to be. You just hired a bunch of people who all had experience typing on blue paper. Typing however, was not their specialty. Research was not their specialty. RNAi was.

Biotechnology has been reduced to mostly monoclonal antibody companies. They sometimes try to break up the molecule or make it more stable but these are bells and whistles that don't work very well. The business model remains, select target, make antibody, start clinical trials, partner or be acquired by big pharma. It's all just typing on blue paper specialists. Basic research, starting from a known ignorance, is the only way innovation will take place. Who has the courage to admit that they know how to type but they don't have any experience typing on blue paper?

Dendreons Four Extra Months

My wealth is all at your command
If you will move your icy hand

Dendreon heard the plea and they made a deal with death. But death is a son of a bitch. Now Dendreon is not going to be the next billion dollar biotech in the Northwest. The projection of $350 to $400 million in earnings for 2011 was a little off. It now looks like the number will be below $200 million.

What the leaders didn't count on was a lack of interest in what they were offering. Dendreon diagnosed their situation and found that the problem is primarily happening with small community-based physicians. As leaders of the biggest Cargo Cult in the Northwest, they had only been associating with "top academic centers that have been familiar with the product for years in clinical trials." They are having a hard time getting the dumb hick doctors on board. David Miller, an analyst with Biotech Stock Research in Seattle said, “Docs are not prescribing Provenge until they are certain they are going to be reimbursed.”

One reader of Xconomy had a slightly different version of the predicament Dendreon is in:

As an oncology practice administrator other than the high cost another issue is the data itself- patients may be hesitant to use a drug with 4 month survival when there are other options available. And if medicare remibursement decreases to ASP + 4 next year, these very expensive drugs could be difficult to justify administering in the community setting

Mitch Gold, the CEO of Dendreon said that the company needs to educate physicians about how the reimbursement process has been streamlined. The education in this case goes in the opposite direction. The market has educated Mitch Gold and the investors.
In their arrogance, they forgot that the patient has a say in their health care. It's not just a paycheck for Dendreon, it's an end of life decision with options.

For the investors, the education came from the earnings call this week. For Mitch Gold and the insiders of Dendreon, the education took place on a daily basis. They just didn'tshare it with the investors. Someone asked the question back in March, "Why is Mitch selling his shares?" We have an answer.

When a company is dying, you don't pump more money into saving it. Clearly, the insiders at Dendreon knew they were in trouble.

Research Department

My mom called with an idea. Her neighbors son was working down at the local university in the "Research Department". Why don't I apply and move closer to home?

The Research Department?

What Feynman was talking about in Cargo Cult Science, was research. He did not condemn all of psychology for flawed research methods. He spoke specifically to a problem in their experimental design. Not all people can spot experimental flaws, but it can be taught. For some this course would be easier than it is for others. Take for example, a recent Huffington Post article looking at the divorce rate for childless vs. couples with kids. It was mentioned that in 1950:

For couples without children, the divorce rate in 1948 was 15.3 per 1,000. Where one child was present, the estimate rate was 11.6 per 1,000. The figure thus continues to decrease, and in families with four or more children, it was 4.6. Altogether, the rate for couples with children was 8.8 per 1,000. In other words, the rate for childless couples was almost double the rate for families with children.

Over 2,000 comments turned up with people chiming in with their views on kids and marraige. One person however called attention to the notion that only 11 to 15 people per 1,000 were getting divorced. The divorce rate is currently around 50%.

The bias here got everyone talking about marriage and family. The "Research Department" guy picked up on a research issue.

Why don't they teach courses on how research is conducted. We can teach chemistry, biology psychology, and we assume that they understand where all of that information came from. Why don't we specifically get at the issue of research? In another HuffPo article a Gender Studies professor asked the question of whether or not men are what they used to be. She mentioned a study where men and women were asked questions about family and marriage. Once again, the comments section found people focusing on what makes a man a man. There were however those who brought up the fact that no such questionnaire was given to people in a previous era.

By all means, every University needs to establish a research department. Before a grad student begins conducting research for his/her professor, they need to do research without the bias they will encounter. Give students the tools they will need when facing a Bulfone-Paus or David Baltimore.

Talent vs Experience

I believe it is more important to hire for talent than it is for experience.

Bill Biggs, President of Li Cor

Talent: A special natural ability or aptitude

Experience: A particular instance of personally encountering or undergoing something

What are the lessons from the rise and fall of SIRNA? J. Michael French, former Sr. V.P. of Corporate Development at SIRNA, is currently the CEO of Marina Biotech, another RNAi company. Barry Poliski, former Chief Scientific Officer of SIRNA, is now the Chief Scientific Officer of Marina Biotech. One might think that Marina Biotech is in for a slow painful death. Can these executives make their RNAi company a success? They made Sirna a success story in 2006. Sirna made that success questionable.

How do you separate talent from experience from the pool of RNAi people? In an environment such as biotech R&D, obedient workers are highly desirable. I know a researcher who once worked for Marina when it was called Nastech. This PhD scientist was to use RNAi to knock out one of biotechs favorite targets, TNF alpha. In a one-off experiment, RNAi appeared to reduce joint swelling in a set of three mice (pos control, neg control, RNAi). In the next experiment 300 mice were put to the test. At the end of the treatment each mouse was euthanized and put into a jar with formaldehyde. Without any actual measurements it was clear that the experiment did not reproduce the original results. No measurements were taken. The experiment at this point ended. 300 jars, each containing a dead mouse sat under an unused bench space for a year. The PhD who ran the experiment was a smart person who knew the honest approach would be to take the measurements. But he was also a new father and he needed to keep the job. Contributing to a massive pile of failed RNAi experiments would be detrimental to his career. Whatever scientific talent he had (the ability to bend over backwards to prove yourself wrong, to report all results...) had to be put aside.

Any person who has worked with RNAi can tell stories like this. It is the classic beer and pizza diet research project. The obedient worker is told to go into the lab, put small pieces of RNA in a plate of cells or a mouse and come back with results that indicate the knock out effects of RNAi. Long before the delivery issue was adopted, researchers were pointing fingers at whoever last touched the RNA. After trying to beat that square peg through a round hole, they finally decided to use a chemistry approach to getting the RNA to behave. Nucleic acid analogues, changes in formulation, injection techniques and many more ideas have come and gone. What remains is the decision that it is the delivery of the RNA that is preventing the desired action. At this point in the career of an obedient experienced RNAi lab worker, you should be questioning the talent of the decision makers.

Yet when you hear the talk, it can be some of the finest you've heard. Complex systems are eloquently spelled out with limitations being overcome by clever ideas. The talent is in the discussion. Experienced scientists are accumulating more and more data that is being processed by a talented scientific advisory board that will lead to the successful completion of a RNAi drug development program. We still have Alnylam, Marina and many more smaller players trying to make this work. They have plenty of experience in RNAi research. Do they have the special natural ability/aptitude to translate what they've experienced into a successful drug development project?

Is It Me or Is the Emperor Naked?

It's time to take another look at the Cargo Cult of RNAi research.

Merck bought Sirna back in 2006 for 1.1 billion U.S. dollars. They announced last week that the company will be eliminating 13,000 jobs, 12 percent to 13 percent of jobs by 2015. Merck also announced an end to its Sirna unit.

I first started blogging about siRNA research back in June of 2006. I mentioned the Merck deal in March of 2006. It is now July 30, 2011. Roche has given up. Pfizer gave up. Now Merck, while not yet admitting that the Emperor has no clothes, is making significant cuts and shutting the doors of its main RNAi investment research facility.

It's easy to be the Cargo Cult Scientist and use my considerable skills to predict the future. A pessimist is more often correct than the optimist but that is not my secret. I've done the research. I have spoken directly with the leaders and I have studied their ways. Why was RNAi selected by the leaders to be the next big thing?

The leaders of the Cults need to tackle problems that they can handle. Curing disease and helping people is tough. Making drugs is a bit easier. They begin with a Drunk Under the Streetlight approach. RNAi is an easy choice for this approach.

How do they direct the lab people to do actual laboratory research in RNAi? Beer and Pizza diet!

At some point some naive little person is going to point out that the Emperor has no clothes. How are they going to handle that?

An Emperor who cares for nothing but his appearance and attire hires two tailors who promise him the finest suit of clothes from a fabric invisible to anyone who is unfit for his position or "just hopelessly stupid". The Emperor cannot see the cloth himself, but pretends that he can for fear of appearing unfit for his position; his ministers do the same. When the swindlers report that the suit is finished, they mime dressing him and the Emperor then marches in procession before his subjects, who play along with the pretense. Suddenly, a child in the crowd, too young to understand the desirability of keeping up the pretense, blurts out that the Emperor is wearing nothing at all and the cry is taken up by others. The Emperor cringes, suspecting the assertion is true, but holds himself up proudly and continues the procession.

The Emperors of the RNAi cult are cringing more than ever. But they hold themselves up proudly and continue the procession.

Still, McConnell said Merck continues to invest significantly in RNAi technology. The decision to close the facility was based on other factors, including ongoing efforts to manage fixed costs. The technology Merck acquired in its acquisition of Sirna has been integrated across the company’s R&D organization, he said.

I succeed at predicting the future because I have looked into the eyes of the leaders and I know how Cargo Cults operate. I know that a scholarly scientific look into the research of RNAi will not appear because it fails to promise the coming of the cargo, only reasons as to why it did not come. We continue to look to the skies.

Cetero Response to Form 483

A spokesperson for Cetero Research sent Science Business the following statement:

Cetero Research, the leading provider of early phase research services, remains fully committed to maintaining the quality and integrity of the data collected in each of its facilities, including the Houston, TX, bioanalytical laboratory. It is this commitment that makes the broad action announced publicly by FDA on July 26, 2011, even more difficult to understand.

Cetero initiated its own internal investigation of its Houston bioanalytical laboratory over two years ago when it discovered the recording of inaccurate day/time data by a small number of research chemists in its Houston facility. Cetero proactively contacted the FDA to self-report its preliminary findings, as well as seek agency feedback on its comprehensive investigation plan.

At that time, Cetero clients were also contacted to make them aware of the situation. The Untitled Letter does not accept the results of our rigorous scientific analysis and discredits the Company’s 1,200 dedicated and experienced employees. The research conducted on behalf of our pharmaceutical sponsors can be, and has been, properly validated.

The FDA said in its public announcement: “It is unlikely that these concerns relating to data integrity affect the overall safety and efficacy of drugs already on the market and, at this time, there is no evidence of problems with the safety, quality, purity or potency of drugs already approved.” Thus, the FDA has not questioned the safety or efficacy of drugs already approved, marketed, and based on data generated from Cetero’s Houston bioanalytical laboratory.

We are pleased that we will now have the opportunity for the first time since our voluntary disclosure was filed in June 2009 to meet with FDA decision-makers to seek to resolve this matter in an appropriate manner. Cetero will continue to cooperate fully with FDA and with our clients and support them through this confusing process.

The little guys are going under the bus! If the FDA had any scientific proclivity they would immediately and randomly audit another group working in the sample group of "the Company’s 1,200 dedicated and experienced employees". As for the narrow window Cetero speaks of in this response, the FDA is looking into a five year period between April 2005 and June 2010.

As noted in a letter FDA sent to the company, Cetero also failed to conduct an adequate internal investigation to determine the extent and impact of the violations and failed to take sufficient measures to assure data integrity within the 5 year time frame.

FDAs Form 483 for Cetero

Just an image. Poor quality I know but I wanted to post this rare recognition of a Cargo Cult Organisation. I don't think Cetero is unusual in its dishonesty but it is unusual that they were caught.

Cetero

Oh my!

The Food and Drug Administration said on Tuesday two 2010 inspections, an internal company investigation and a third-party audit uncovered "significant instances of misconduct and violations" at a Cetero facility in Houston.

The Cary, North Carolina-based firm does early-phase clinical research and bioanalytics for a number of drugmakers. The pharmaceutical companies can then use those studies as supporting evidence in drug approval applications to the FDA.

What if this type of inspection became routine for all of science? Randomly pick a NIH grant recipient, a biotech company, or a pharmaceutical company and ask them how they came to their conclusions.

“When someone says, ‘Science teaches such and such,’ he is using the word incorrectly. Science doesn’t teach anything; experience teaches it. If they say to you, ‘Science has shown such and such,’ you should ask, ‘How does science show it? How did the scientists find out? How? What? Where?’ It should not be ‘science has shown.’ And you have as much right as anyone else, upon hearing about the experiments (but be patient and listen to all the evidence) to judge whether a sensible conclusion has been arrived at.

What we have is a pandemic of scientific misconduct. Cetero, Cargo Cult Contract Research Organisation!

Malcolm Gladwell

"You don't start at the top if you want to find the story. You start in the middle, because it's the people in the middle who do the actual work in the world," writes Malcolm Gladwell in the preface to WHAT THE DOG SAW.

"How do we know whether someone is bad, or smart, or capable of doing something really well?" he asks in his book 'Blink'. In this book he looks into how educators evaluate young teachers, how the FBI profiles criminals, how job interviewers form snap judgments.

In 'Outliers' he tries to figure out why some people are successful. He comes to the conclusion that we've focused too much on the individual and failed to consider the other factors around successful people. We fail to see the forest for the trees.

Interestingly he gives an example of successful people. Jewish men who grew up during the depression whose fathers worked in the garment district apparently have done quite well as corporate lawyers in New York City. Our hero R. Feynmen also fits this category.

'The Tipping Point' explores meaningful changes and what really brings them about.

I've struggled all my life to understand this world. I got into the science business to seek refuge from a demon haunted world. Things were no different. There is of course the beauty of real science that occasionally works its way into the mess that professional scientists have created. But the world seemed even more bizarre watching people with PhDs use science in the manner that they do. Professional scientists are people who want to be known as smart. They want to be experts. Actual scientific people would never devise a thing such as the peer reviewed journal. RNAi does not come from scientific minds. Biotechnology hasn't failed for no reason.

This blog wonders how a group of highly educated individuals with billions of dollars create the system of scientific research that now passes for "discovery"? We are still standing on the shoulders of giants, but we seem to be looking in the wrong direction.

Along my journey I have extracted ideas from people and things I've read and observed. These ideas tend to follow what is considered to be the scientific method. That doesn't mean the ideas have always come from scientists. Malcolm Gladwell for example is a journalist. People who are good at what they do have the ability to see what matters. Gladwell applies new questions to old problems. Why are some people, of equal ability, more successful than others? When do good ideas (and bad) become accepted? To those of us who are dissatisfied with current explanations, this is a breath of fresh air. The books serve two purposes. First to see new explanations to old problems and, more importantly, to take the journey of how they were obtained.

Malcolm G. is not without his own flaws however. In the future I hope that he re-visits the following ideas posted on his website:

Journalism is not like the business world, where the mechanics of decisions and procedures take place behind closed doors. It is, rather, like science, where the fruits of all endeavor are put on public display. In the world of science, that transparency allows the profession to be self-policing. It is very hard to commit scientific fraud because all significant findings are published, scrutinized by other members of the scientific community, and—if they are sufficiently controversial—independently tested.

No one's perfect.

Tumor Size

A tumor, as described by Wikipedia, is commonly used as a synonym for a neoplasm (a solid or fluid-filled (cystic) lesion that may or may not be formed by an abnormal growth of neoplastic cells) that appears enlarged in size. Tumor is not synonymous with cancer. While cancer is by definition malignant, a tumor can be benign, pre-malignant, or malignant, or can represent a lesion with no cancerous potential whatsoever.

What does the size of a tumor have to do with cancer and health? It is possible that we often develop tumors. Without a doctor around to tell us what they are, our body deals with them and we are none the wiser. Some tumors however grow and make people quite uncomfortable, such as the picture on Wikipedia. Besides discomfort, a tumor can be a threat to your health.

In 1971 Dr. Juhah Folkman formulated the idea of tumors being dependent on blood vessel formation. Researchers eventually warmed up to the idea and started looking for angiogenic factors that could be target by drugs. VEGF, Vascular endothelial growth factor became a target and it worked, some of the time. In cancers deliberately developed on the backs of mice, for example, blocking VEGF prevents tumors from growing. Clinical trials on human beings showed that it adds time to a cancer patients life when used in conjunction with chemotherapy on colon and non-small cell lung cancer. However, it did not add time to a patients if they had breast cancer. But science isn't always straight forward.

The FDA approved Bevacizumab in 2008 for use in breast cancer. A panel of outside advisers voted 5 to 4 against approval, but their recommendations were overruled. The panel expressed concern that data from the open label clinical trial did not show any increase in quality of life or prolonging of life. The trial did show that Bevacizumab reduced tumor volumes and showed an increase in progression free survival time. Based on this data the FDA chose to overrule the recommendation of the panel of advisers.

Two subsequent double blind studies showed little efficacy and some serious side effects. The first rule in medicine is to do no harm. It is a rule that is being ignored by scientists and doctors at Roche and Genentech. Careerism has them continuing to tout the drug in spite of the trial data. It may help some people just as it might result in the many side effects. Prescribing the drug by doctors (which they can still do) would be a random act. If the patient gets better... Eureka! If they get the side effects or they die on schedule, it was an act of compassion. None if it could be considered science. At $90,000 a year, Avastin is a money maker. It is the number one selling cancer drug in the world bringing in around 6 billion dollars a year. Removing the breast cancer indication will cut profits by a billion dollars.

Tumor size is biotech endpoint. Shrinking tumors means making money. The real question is whether or not it helps cancer patients feel better and live longer.

The Value of Blogging

While reading David H. Freedmans book, "Wrong", I knew he was preaching to the choir. But what would someone like my mother, a non-skeptic, think about the book? She needs to believe that our chosen experts are reliable. When something happens such as the Casey Anthony verdict she becomes a skeptic of the legal system. When the damned liberals take control of the white house she becomes a skeptic. More on that line of thought later. In general however, the non-skeptic does not like books like 'Wrong'.

Being accepted as an expert takes time. You have to build up credentials like college degrees, years of experience, and you have to be well spoken. The credentials quite often trump real expertise. Take for example the GETRAPL story. Using the 'drunk under the streetlight' approach the experts found GETRAPL. They are wrong. But who will take the word of a blogger who has never given any credentials and who seems to be an angry person?

My non-skeptical mother doesn't like angry people. Not many people do. She prefers the well spoken, soft spoken, expert who affirms her own beliefs. The same could be said about myself and the author of 'Wrong'. But I differ in that I don't care what the credentials of D.H. Freedman are. I like the ideas he presents. As a skeptic, I would like to think that I could tell the difference between right and wrong based on my experience, not the background of the person to whom I'm listening.

To what extent then do I, the Cargo Cult Scientist, expect to be taken seriously as a contributor to what is right and what is wrong? I don't. I want the words I write to be considered but I don't want to be an expert. I am not willing to go on camera and give away my identity. I have not spoken kindly and without anger about my chosen profession. The notoriety I could achieve would certainly focus on my anger, and weak credentials. I would be labeled a fool. As pointed out in 'Wrong':

It has long been established that when people are part of a crowd in which they're anonymous, they tend to behave less conscientiously than when they're identifiable.

Blogging about the how wrong other people are is cowardly. Yet the track record of the industry is hard to dismiss. Many people have sang the same song, Biotech is a failure. Working with the people has left me dazed and confused about what science is suppose to accomplish.

On the surface, a brave approach to the discussing the biotech business would be to start a website and actually talk to the leaders of the industry. Xconomy is such a website. Just yesterday an article was posted regarding the troubles facing Biotechs inability to attract the kind of financing it did back in its glory days. A lack of guts seems to be an issue. The author of the article, Luke Timmerman, is a well respected member of the biotech business. Luke didn't start a blog. He started an online website dedicated high tech news that is meant to make money. Telling investors they are lacking guts, takes guts. Luke also posted an article telling college students to pursue computer science instead of biotechnology. That takes guts. Luke finds himself in front of the crowd, conscientiously discussing their work. He must be smart and not come across as I do.

What then is the value in this blog? The value of the Cargo Cult Scientist has been mostly therapeutic. The daily struggle of listening to people with all the right credentials and millions of dollars is difficult. They are in the right place. But are the right? According to the ideas of Cargo Cult Science and numerous books such as 'Wrong' they are not. Being right is often the wrong thing to cling to but in science, it is the only thing that matters. I blog because I think science is a beautiful abstract concept, just like Feynman laid out for us in CCS. It makes me feel better.

The Same People

When you invest in biotechnology, you are investing in people. The Cargo Cult you invest in is a group individuals who will be spending your money.

H. Stewart Parker is a 55 year old seasoned veteran of the cargo cults of Seattle. She's a pioneer. She was the first employee at Immunex right around the beginning of biotechnology. She went on to be the CEO of Targeted Genetics, a spinoff of Immunex. Before her reign ended, Targeted had ran up a deficit of $310 million. She left her cocoon in 2008 to become a consultant, as most execs do after their companies tank. She moved on to WBBA, the local biotech promoting organization of the Northwest. Today she announces an end to that moonlighting job so she can work full time as the CEO of the Infectious Disease Research Institute.

Let's break down her place in the Seattle field of leaders. The nonprofit she'll head up was founded by immunologist Steve Reed in 1993. Steve Reed is the Executive VP of Immune Design, and co-founder of Corixa. Bruce Carter, former CEO of Zymogenetics, is the Chairman of the Board for Immune Design. Other Corixa co-founders include Ken Grabstein of IL-15 fame and CSO of local Allozyne and Steven Gillis, co-founder of Immunex. Steven Gillis hired Ms. Parker straight out of college (UW). The names of the companies may change but the people (who spend your money) remain the same. They are all still here, running companies and enjoying the excess of the massive funding that comes and goes in the cargo cults.

This group has done well for themselves but have done little in terms of creating a career path for scientists. The business people have the job of telling the story that is intended to be told from the inception of the company. That is the career path that Seattle biotech cargo cults nurtures and structures. The career of the scientists is to come in as needed and fill in a piece of the puzzle and then go away. If a scientist can't tell the story, they can easily be replaced.

How can we find success in biotechnology when the leadership seems to be making it all up as they go along? Ms. Parker has had an experience. Since the beginning of biotechnology she has worked as a leader. She remains a leader. She has experience. But unlike a winning Superbowl coach, her experience will not land her a book deal on managing a winning team. But it will get her another CEO job in Seattle.

The IL 15 Cargo

Science is an abstract concept. In describing the cargo cults Feynman pointed out the difficulty of defining what science is.

But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school--we never say explicitly what this is, but just hope that you catch on by all the examples of scientific investigation. It is interesting, therefore, to bring it out now and speak of it explicitly. It's a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty

How has the scientific community dealt with the IL-15 story?

if you're doing an experiment, you should report everything that you think might make it invalid--not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you've eliminated by some other experiment, and how they worked--to make sure the other fellow can tell they have been eliminated.

One thing that casts doubt over the IL-15 story is that people seem to have a hard time finding it. N-Rays were hard to find so we looked into it. There are those who have created a convincing narrative to describe the life and times of IL-15. The problem is that recently there have been an inordinate amount of retractions of papers that created the narrative. Silvia Bulfone-Paus has had 12 papers retracted. The reason was fabricated data obtained by two rogue laboratory people. Problem solved? Not quite. Six more papers are under investigation at a second University where the Silvia Bulfone-Paus also works. The rogue white lab coat wearing scoundrels, Elena Bulanova and Vadim Budagian, weren't around during this research!

Why so much controversy? There appears to be a discrepancy between the IL-15 narrative and its ability to predict what will happen in the laboratory. Rather than presenting the puzzling data, they altered the results to fit the narrative. They were caught. Twelve papers and counting have been taken out.

Is IL 15 real? You can buy the protein from R&D Systems. There are antibodies against it. It was cloned in 1995. But a real scientific approach to understanding this protein would start by going into the laboratory to reproduce some critical work. Start by cloning the protein again. Does the DNA sequence match? Do the antibodies available bind to the newly cloned IL 15? Does the purified protein mimic IL-2–induced T-cell proliferation? The basics would be a simple place to start, accumulate IL 15 and begin to examine the role of this protein.

Another way to examine the role of IL 15 is to look into the careers of the researchers. We know there are a couple researchers who wish they had never met IL 15. The lead author of the first paper describing IL 15, Kenneth Grabstein, is now the CSO at Seattle based Allozyne, an Accelerator company that recently bought Poiniard indicating their dedication to business as opposed to science. This company is a tiny little place one block away from Accelerator that has a singular mission to merge and make money for its investors. When did Dr. Grabstein move on from IL 15? In the world of Cargo Cults Seattle, Dr. Grabstein has done quite well for himself. However, he has left path of abandoned research for others to sort out while cashing in on companies such as Immunex and Corixa. Immunex made its fortune from TNF alpha. Corixa was acquired by GSK. IL 15 provided Dr. Grabstein some early publications but later in his career he lost interest in helping the world understand this protein.

You would think that the laboratory approach to clearing up some of the mystery would be employed by someone. Make a list of things that were fabricated, such as the many western blot images. Simply redo them and publish your results. Your conclusions would be simple, "We do (or do not) see a band on this western blot. Keep the narrative out of it. Make a list of repeated experiments and provide your results. Don't interpret them, just provide the results. While the office-bound professors and early-pioneers-now-turned-businessmen hope it will all blow away, lets hope it doesn't.