The Impact Factor

What is most interesting about science is that it is a way of thinking that leads to the truth. It helps everyone view the world as it really is, not as we want it to be. After basic understanding is acheived, we can use the knowledge to develop things like light bulbs and cures for disease. We call that technology. The opposite of this process is religion. Religious individuals see the world as they want it to be. Opposing views on the meaning of things leads to conflict.

That is why I write this blog. When people hijack the good name of science and have lucrative successful careers in science we end up with conflict. That is why "The Impact Factor" is high up on my radar. The need professional scientists have to impact others must not get in the way of the need scientists have for the truth. I believe that even professional scientists can cross over to religious thinking, finding meaning where none exists. The result is not useful technology but rather, conflict.

In "The Impact Factor" posted on Retraction Watch, there is an interesting juxtaposition of two different views of how professional scientists operate.

Commenter #1 - John: There is also faking data…. which is the riskier, but also successful…and immoral/unethical. 

Commenter #2 - Chris: That is terribly cynical John... The notion that “faking data” is a successful approach to publishing in high impact journals is astonishing. Try it and see where that leads you in the long term!

You can probably guess where my bias is on this topic. I agree with John. Am I viewing it through a religious or scientific perspective? Is Chris religious or scientific in his views?

Chris has taken the religious view of science in this case. I fully understand his opinion that "faking data" is a poor choice. I've dedicated this blog to the notion of Cargo Cult Science. Faking data is a necessity to keep a Cargo Cult in operation. Where Chris and I part ways is in thinking that getting published is the same as thinking scientifically. Let's break the two up and look at Chris' statement.

If you want to do science and build one idea upon the last, you cannot cheat. The analogy I have used is a crossword puzzle. If you fill in a word improperly, the words that cross the incorrect word will be harder to figure out. The certainty you have of the crossing words will lead you to change to incorrect word. If you are like me, you often get words wrong but you always have a certainty value in your head. If the horizontal word has a high certainty value, you will have a lower certainty value for your proposed vertical crossing word. This orthogonal approach to solving questions increases the certainty you have over singular unsupported ideas. In science we have citations and other scientists who validate our work to increase certainty.

Getting published however only validates your ability to get published. It indicates that you know the game and you play it well. Like a lawyer who has only to convince the jury, the scientist has only to convince a few people at the journal. The journal lays out its criteria so the scientist knows ahead of time what kinds of results will get him/her published. Quite often the result coming out of the lab put a damper on your hopes and dreams. So cheating creeps into ones mind, and as we know, some scientists will go ahead and cheat. Just read the articles above "The Impact Factor" for a few examples of people who cheated, advanced their careers, then got caught. Nitin Aggarwal cheated on his PhD thesis, got his Phd, won a $1,000 award for that thesis and landed a job at BMS. Chris points out, "see where that leads you in the long term!" Chris failed to chooses to ignore these short term advantages.

As much as we want believe that cheating doesn't happen, and that scientific journals cannot be fooled, we have to face the facts. Much of what gets published is unreliable. Much of what gets published is not reproducible. That is not a negative viewpoint. If we can weed out the unreliable and non-reproducible, we will end up with more useful technologies. Therefore, what many people like Chris see as a cynical view, is really a scientific view. It is scientific because the truth is what matters most. Our  opinion of the truth is immaterial. As the human race progresses, scientists will have their impact. The impact that the truth has on the scientific community is what will have the greatest impact on the human race. For the likes of Chris to close their eyes and hold those with eyes wide open in contempt is a religious viewpoint. If we assume that the cheaters will all get caught, we have not been paying attention to what we see in the journals. We have not payed attention to the biopharma industry.  Cheating is a tool used by fools and knaves. The fact the so many fools and knaves succeed in the short term, and some in the long (Don Polderman) should tell us that we need a change.

PCSK9 Drugs

Sanofi and Regeneron report "POSITIVE RESULTS" with Alirocumab (anti-PCSK9) from their first phase III clinical trial for LDL Cholesterol Reduction. LDL is the bad cholesterol, HDL is the good. These terms are unfortunate since the body has deemed both forms as necessary. Over time the body may build up LDL which is an indication that you are not regulating your diet and exercise properly. LDL cholesterol can build up on the walls of your arteries and increase your risk of heart disease. However we have to wonder how to interpret the accepted medical stance on cholesterol. 

In Good Calories, Bad Calories, Gary Taubes gives a history of cholesterol research. Not everyone agrees with the current information on cholesterol levels and the role cholesterol plays. The way in which some minds work is that if something is bad, it should be eliminated. That works fine if you are talking about crime in your neighborhood. But LDL is there for a reason. It is coded into our DNA. The question then is the range in which a healthy individual maintains LDL levels. If BigPharma wants to make drugs to help sick people mimic healthy people with regards to LDL levels, what is their goal? 

The mean low-density lipoprotein cholesterol (LDL-C, or bad cholesterol) reduction from baseline to week 24, the primary efficacy endpoint of the study, was significantly greater in patients randomized to ezetimibe (47.2% vs. 15.6, p < 0.0001)

Back to the concept of an accepted range. If their is a level too high there is a level too low. What does the 47.2% reduction in LDL from alirocumab vs 15.6% reduction from ezitimibe (Zetia) mean? One goal of Regenerons research is to determine  the lowest effective dose of anti-PCSK9 antibody. The reasons for this are best summed up by these findings.

The percentage of patients who reported treatment emergent adverse events was 78.4% in the ezetimibe group and 69.2 in the alirocumab group. The most common class of adverse events was infections (39.2% with ezetimibe vs 42.3% with alirocumab), which included nasopharyngitis, influenza, and upper respiratory tract infection. 

Like cholesterol, there is the good and the bad. We see that anti-PCSK9 products can lower LDL-C. They did not say if the people felt better or got healthier. They did not say if they could more easily and safely adjust a persons LDL levels into the range found in healthier people. The just lowered the bad. In the process they increased a range of infections that can be deadly to older people.

Check out this article on the dangers of PCSK9 targeted drugs. "New Cholesterol Drug PCSK9 Likely Is to Prematurely Kill You". If lowering you LDL levels is the good news, this has got to be the bad. Dr. Mercola brings up a few topics the Regeneron press release does not. There is a lower limit not to be crossed. Lowering your LDL levels to below 50 is dangerous. Your body needs cholesterol to produce hormones, cell membranes, vitamin D, and bile acids that help you digest fat. The most effective method of regulating your cholesterol profile is through diet and exercise. That's the good news. What about this added downside of dropping dead from anti-PCSK9?

Again we get the lower limit discussion that Regeneron has omitted from their world view of cholesterol. According to Dr. Mercolas' article, low HDL can lead to poor memory and a decline in memory for middle aged people. You may also experience violent behavior and aggression,  depression, suicide, cancer, Parkinsons disease, and heart disease. Dr. Mercola states that the odds are greater than 100 to 1, that if you are taking a statin to lower your cholesterol, you really don't need it. Statins are what the PCSK9 drugs are going to compete against.

Statin drugs, which one in four adults over 45 are using to protect their heart health, can paradoxically have significantly detrimental effects on your heart health 

Our understanding (our science) of cholesterol has been touched by the Cargo Cults. We have to look into these things that don't work and ask why. Not only do we muddy the waters with bad and biased pharmaceutical research, we create dangerous products for the most vulnerable. I think it is fairly clear that cholesterol research needs to be revisited, new guidelines need to be established and diet and exercise needs to be advertised. PCSK9 should be studied very carefully to make sure it does not reduce your own ability to fight heart disease and worse, to simply kill you. In my Utopia, all future studies would include a diet and exercise arm. If that were the case I am sure they would halt the study and never again allow a statin or anti-PCSK9 enter the body of a person capable of eating healthier and increasing their level of exercise.

Serepta Clarification

I saw a few comments on an investor website concerning my post on RiAN BioSciences and the Disappearing Bees.  It was accurately pointed out that I erroneously labeled Serepta as an RNAi company. I need to clear up my mistake. They are not an RNAi company. Their drug is a non-traditional RNA antisense product. They point out, "Unlike most other RNA-based approaches, such as small interfering RNA (siRNA), Serepta's technology can directly target both messenger RNA (mRNA) and precursor messenger RNA (pre-mRNA) to either down-regulate or up-regulate targeted genes or proteins."

I simply lump all RNA drug companies together because I think targeting RNA is a mistake. I stand by that claim. Serepta, like the others, has some interesting chemistry in its product. Phosphorodiamidate morpholino oligomers (PMOs) are synthetic structures modeled after the natural nucleic acid framework of RNA. They replace the ribose ring with morpholine which is linked through phosphorodiamidate rather than anionic phosphodiester bonds. You can get some of this stuff from a company called GeneTools LLC and start your own RNA company or just do some research with it.

Gene Tools, LLC produces and markets Morpholino oligos for antisense applications and ancillary products for basic research. We are also engaged in research relating to delivery of Morpholino oligos into cells and use of Morpholino oligos as probes and in diagnostic systems.

Morpholino oligos and the subunits used in their assembly were invented and developed at Sarepta Therapeutics (formerly AVI BioPharma Inc. and earlier ANTIVIRALS Inc.), the pioneer antisense company founded by James Summerton in 1980. Gene Tools manufactures and markets research quantities of Morpholino oligos. In Japan, Funakoshi Co. is the distributor of Morpholino oligos and related products. 

Lest people think I am against everything, I want to say that I love this kind of science. It is not unlike the work of one of my heros Gertrude Elion. Here she talks about her work with nucleic acid analogs:

The work became fascinating almost from the very beginning. We were exploring new frontiers, since very little was known about nucleic acid biosynthesis or the enzymes involved with it. I had been assigned quite early to work on the purines and, with the exception of a few deviations into the pteridines and into some other condensed pyrimidine systems, the remainder of my work concentrated almost completely on the purines. Each series of studies was like a mystery story in that we were constantly trying to deduce what the microbiological results meant, with little biochemical information to help us. 

There came a time where Gertrude had to go off the beaten path and deduce what the results meant. She went from chemistry to biology to "what do these results mean?" It is at this same juncture in the research process where GeneTools LLC and  Serepta parted ways. GeneTools LLC represents the science of PMOs versus RNA.  Serepta represents the business of drug approval. Somewhere between the work of GeneTools LLC and Serepta we enter the Cargo Cults.

Why didn't James Summerton make that journey? The powers of his PMOs as a drug, over RNAi molecules, are touted on the Serepta website:

• The ability to target specific gene sequences without non-specific interactions
• Safe at higher doses
• Able to UP and down regulate gene expression 
• Easier to deliver to specific cells
• Resistant to enzymatic degradation

The questions that AVI and Serepta had to answer was "what to do with it?" and "what do these results mean?" Serepta made some pretty astounding claims. Dr. Summertons PMOs have a superiority (as a drug) over the Nobel Prize winning science of RNA interference. 

A PMO is physically a PMO for sure. The mechanism of action is different. It may very well have the superiority over RNAi molecules that Serepta claims. But RNA deep inside a cell is the target. This is where the science gets soft. A Cargo Cult does not concern itself with the complexity of a cell. Gene expression is not within the purview of Serepta management. That is in the job description of a scientist, not a businessman. What management needs from the lab is positive results that can be used to sell their promise of cargo to investors. And they weren't getting it from their people in Oregon and Washington.

Serepta began in Oregon then moved to Washington then to Massachusetts in search of the magical Pixy dust that will turn PMOs into a drug. The inventor of PMOs did not have that Pixy dust nor did anyone in Bothell WA. What was CEO Chris Garabedian looking for when he sacked his Oregon and Washington staff and moved to Cambridge MA? You won't read about it on their website. You will read about PMOs and the work of the common scientists. What about that Pixy dust? What was the knowledge that could only be found in Cambridge MA that puts the 33 year science project on the final leg of it's journey? After 33 years we can now look to the skies and expect the cargo planes to come because some people on the east coast knew what to do with James Summertons PMOs. 

Investors are starting to have their doubts. Sereptas lead candidate is Eteplirsen:

Eteplirsen is a treatment for Duchenne muscular dystrophy, or DMD, patients, a rare genetic muscle-wasting disease caused by the absence of dystrophin, a protein that is necessary for muscle function. The disease affects one in 3,500 boys worldwide and is one of the most common fatal genetic disorders. Currently, there is no approved treatment for DMD.

Eteplirsen works by skipping exon 51, helping resolve a genetic mutation and allowing for the creation of dystrophin. This, in turn, corrects the body's inability to build muscle. Eteplirsen showed promising results in phase 2b studies, which, following a meeting with the FDA in July, led the company to report that the FDA was willing to consider those results as part of an early approval process. As a result, the company indicated its plans to file an application with the FDA in early 2014.

A good deal has changed since the July conversation with the FDA Several things have changed since that announcement. Most important, an announcement came that GlaxoSmithKline's (NYSE: GSK  ) and Prosensa's (NASDAQ: RNA  ) competing exon-skipping drug drisapersen came up short in phase 3 trials. In that trial, drisapersen failed to significantly improve muscle function despite showing considerable promise in earlier trials.

That late-stage failure kicked off a flurry of interest in eteplirsen. Investor hopes for early approval may have jumped the gun, however. Given that Glaxo's drug and eteplirsen similarly produce dystrophin by exon skipping, some are wondering if the FDA will back away from its earlier willingness to solely consider phase 2b data.

What interests the Cargo Cult Scientist the most is the disease. How does it work and are we ready to fix it? Apparently the approach Prosensa took was a little off. What were the details and why then does Serepta have a better shot with the same exon 51 skipping approach? Investors and biotech CEOs are speculators. Scientists find ways of injecting certainty into their predictions of future outcomes. Will morpholinos succeed where RNA failed? We will be watching.

Passion, Dispassion and Pink Ribbons

Raising money has become the priority, regardless of the consequences. 

It's almost as though our disease is being used for people to profit. And that's not okay. 

For people to finally rise up and object, they have to be aware of the lies they are being fed.

One more thing about this Atossa Genetics Cargo Cult. The three quotes above came from the Pink Ribbons Inc. documentary. There is a lot going on here and it's more than just pink ribbons and breast cancer. It represents all of the pharmaceutical industry. Dr. Quay has made himself a millionaire by running biotech/pharma highly questionable companies. He is not a breast cancer specialist. Previously he tried to sell us a nasal spray to fight obesity. He knows that disease is profit and he seeks the most profitable. We as human beings need to guard against profiteers who wish to fill the market with ineffective, poorly researched, poor quality, potentially dangerous products. Open the Atossa Genetics website. Do you see a color similar to the pink used by Susan Koman? Coincidence? The pink ribbon profiteers paved the way for the latest Quay invention. The product is cheap plastic suction cups made in China and a genetic test that the FDA says does not have sufficient quality control. The "National Reference Library for Breast Health" is not federal government institution. It is not a branch of the NIH or the NSF. It is something within the Atossa company. A company that uses the momentum of the money making pink ribbon machine to put more money into the pockets of a few Cargo Cult leaders.

The best way to conduct research is the opposite of what we see with the pink ribbon crowd. The best way to conduct research is to do so dispassionately. If we in the life sciences were to create a manifesto for living the life of a scientist it would have to include the idea of dispassionate research. We can be passionate about our science and our work. We just can't let our passion for anything get in the way of seeing the world for what it really is.

Dispassionate: Devoid of or unaffected by passion, emotion, or bias.

The way in which biotech and pharmaceutical companies get support for their work is by catering to the passion others have in finding cures for disease. At this point of my argument I might be tempted to quote Jonas Salk, "Can you patent the sun". It is a quote often used as the moral impetus to forgo patents and profits from vital medical innovations. To quote from this post from Roy Zwahlen in BiotechNow:

As pointed out by Robert Cook-Deegan at Duke University, “When Jonas Salk asked rhetorically “Would you patent the sun?” during his famous television interview with Edward R. Murrow, he did not mention that the lawyers from the National Foundation for Infantile Paralysis had looked into patenting the Salk Vaccine and concluded that it could not be patented because of prior art – that it would not be considered a patentable invention by standards of the day. Salk implied that the decision was a moral one, but Jane Smith, in her history of the Salk Vaccine, Patenting the Sun, notes that whether or not Salk himself believed what he said to Murrow, the idea of patenting the vaccine had been directly analyzed and the decision was made not to apply for a patent mainly because it would not result in one. We will never know whether the National Foundation on Infantile Paralysis or the University of Pittsburgh would have patented the vaccine if they could, but the simple moral interpretation often applied to this case is simply wrong.”

I used to live close to a Ronald McDonald House and the Seattle Childrens Hospital. Every time I went to the store I parked by the playground next to the Ronald McDonald House apartments. Little children, bald from chemotherapy, would be playing with their families. They just want to have fun and live life but soon you'd see them heading up the street to the hospital for God knows what. Given the opportunity to work in research you could use such an image to passionately pursue a cure. Once you start to look at the data however, you must put your passion aside. You must follow the precepts of the scientific method. It does not allow you to take shortcuts to save that one child whom you've just watched walking to the hospital. Likewise, your desire for fame and fortune should not affect the way you design experiments or interpret the data. It should not effect how you communicate your work.

With that in mind think about the brief history of Atossa Genetics. There is the off-pink color of their website. There is the the fact that 20 days prior to the recall we had this sponsorship of The Third Annual Power of Pink luncheon, fashion show and silent auction. There is also the issues raised in the FDA warning letter. In a dispassionate manner, I would you value Atossas charity towards the pink ribbon money raising movement and more than the quality of the product they intend to sell to women with breast cancer. Would you see their charity as part of their marketing plan? 

That is my passion, seeing the world as it really is. I do not see a negative side to pointing out a negative. A Pink Ribbon Profiteer is a bad thing and pointing that out is good. Poor quality science is a taking all of the money and power out of science. The passion people have towards ending disease leads them to donate money to organizations like Susan G. Koman. It leads them to cheer for the financial success of Dr. Steven Quay. However, we must be careful when dealing with our emotions. Others will use them against us. With science, we have discovered the way around all of this. With Cargo Cult Science the others have discovered a way around science. You can give money to charitable organizations, wear a colorful T-shirt, walk a 5K for the cure and hope something positive happens. You can also watch what they do with the money. Watch out for the Atossas because they want some of that money. 

Atossa Revisited

Atossa Genetics received a warning letter from the FDA back in February in 2013 regarding its breast cancer product. What they had was a suction cup device that would remove an aspirate from the nipple of a breast that would be sent off for breast cancer bio-marker tests. Last Friday, Oct. 4, Atossa announced a voluntary recall of their product. 

I talked about this company back in June of this year. Knowing some of the leadership was my first clue that this was a Cargo Cult Company. The next clue came when Atossa received a Form 483 from the FDA. Later they received a warning letter in response to the CEOs response to the Form 483. The latest action at Atossa Genetics does not contain the snarkiness from previous communications. Something is seriously wrong at Atossa. Let's take a look at what the employees think about working at Atossa, as posted on Glassdoor:

Poor management systems, written procedure is given but always changing that made people confused and frustrated. Anonymous current employee #1 

High salaries to compensate for working with high maintenance management. Poor business leadership and no reliable company plan leads to sporadic projects and mis-handled accounts. Anonymous current employee #2 

Gained good work experience and learned how to work with very difficult personalities. People quit and get fired left and right, there is no company culture and relationships between employees are discouraged because of management insecurity. People are only hired into management positions because they are friends or family of the CEO. VERY poor management structure, product is a lie (does not do what it claims to, cannot detect cancer) and FDA constantly writes letters to company about non-compliance. Anonymous current employee #3

Management is not getting a big thumbs up here. One thing I can add is that the CSO of the company is the wife of the CEO. She was a low level researcher at Nastech when she married Quay the CEO of Nastech and now the CEO of Atossa. This is not the way to hire a CSO. Had she missed out on the Nastech job, which almost happened, she may have gone the way of so many ex-Nastechians who haven't worked in biotech since Nastechs demise several years ago.

Atossa is now being investigated for possible violations of federal securities laws by Block & Leviton LLP and Finkelstein Thompson LLP. Just knowing about the management and the FDA troubles should have been enough to convince an investor to pass on this one. What other tricks could Quay and company have had up their sleeves? It is always a difficult path to try and lead others when you are unsure of what you are doing. According to the disgruntled current anonymous employees, management at Atossa is a problem. The FDA has come down hard on the company. In what other ways could Atossa directors have violated federal securities laws or breached their fiduciary duties to their shareholders?  If these lawyers come to the same conclusions that the FDA did back in early 2013, will that be enough to drive a stake through the heart of Atossa?

Going back to Quays previous Cargo Cult debacle, you can see that he has a talent for getting people excited about a project. Check out this unbridled optimism from Jim Cramers Mad Money. "When it coms to biotech, Cramer has generally stuck with three names: Celgene, Genzyme and Gilead. But he thinks Nastech has more ways to win than any of the others, and he likes it here along with the others." That was April of 2007. Then came October of 2007. Nastech failed to earn a $5M milestone payment from big pharma partner Procter and Gamble. Dr. Quay assured Cramer that everything was okay, not to worry. One month later Procter and Gamble bailed out, Nastech stock lost 40% of its value in a day and Cramer had to face facts. He was wrong. 

This is a blog about the Cargo Cults of Biotechnology and the Pharmaceutical business. In this world we have characters who thrive and make lots of money running fake scientific organizations. The people they fool are investors and more than a few tribesmen. Those who invest a chunk of time from their own careers take a hit as well. Cargo Cults are bad for your career unless you know how to win, like Dr. Quay. He isn't failing at what he does. He makes lots of money. Your money. If you want to make a profit by investing in a biotechnology/pharmaceutical company it must be remembered that this is a highly random business that attracts more than a few charlatans. The CEOs, the VCs, the Xconomists, the Bruce Booths, and many others are doing well in this world. How do they do it? Who keeps funding these organizations and their high paid leaders?

Kurt Shilling tried to run a high tech company. He was a baseball player, not a businessman to be put in charge of a multi-million dollar company. He used his own money and now he's broke. Like Shilling, biotech leaders are trained in something other than business and management. They spend years learning one thing then switch to become businessmen. The difference between Shilling and the biotech leaders however is that the latter knows better than to use their own money. Dr. Quay has been at the helm of several companies that lost far more than Shillings company. He keeps coming back. Biotech leaders like Quay pay themselves with other peoples money and hope to get out before the day of reckoning. When they fail you will find them standing right there during the reckoning, denying any culpability. Atossa will be submitting new premarket application by the end of the month, covering the  collection, preparation and processing of breast fluid specimens. Was last Friday the day of reckoning for Atossa? The saga continues but it is more than just Atossa. It is a saga of a leader who creates jobs and builds excitement in the biotechnology business. Once again this leader has directed our eyes to the sky but no cargo came. Just the FDA and the lawyers showed up, wanting to shut the airport down, just like the others before. 

Peer Review Quality Assurance

Peer review does not employ the scientific method. Rather it is a system of endowing a handful of individuals with the authority to validate other scientists research without actually reproducing their claims. The system allows for the publishing of roughly 90% false information. It also creates the need for many to embellish in order to impress the journals. The journals and the peers are looking for the next big thing which is also not exactly what science is. More important to science is the truth. 90% false information is what happens when you start thinking too big. It happens when you start embellishing and over-interpreting your results. A peer review, publish-or-perish world is a recipe for disaster.

The last post was about an incomplete quality control evaluation of the scientific publication business. John Bohannon, science writer and PhD in biology, created a fake paper that should have been rejected. Instead the paper received the thumbs up in 157 of 304 open access journals. The next step of this experiment would be to create a fake paper to submit to paywall-protected, subscription-required scientific journals. Sentimentality or sympathy for the subjects hurt feelings should not prevent us from finding out the truth. The question we have started to work on is the quality control/assurance of published scientific research. The system is in need of an upgrade.

We have a quality control problem. The Amgen Study and the work of Dr. Ionnidis pointed out long before the John Bohannon sting operation that all journals have a quality problem. The Bohannon experiment was meant to enhance the image of the more prestigious journals but the lack of a comparison of the open access journals to Science/Cell/Nature left us to merely assume that Science/Cell/Nature are better. We can't make a judgement on a journals reputation however. It is the system that comes into question. Granted, Science/Cell/Nature will have better peer reviews, they are still merely peer reviews of a few anonymous peers. Since the more prestigious journals are sensitive, how could we study them without tricking them into showing us how bad they are too?

Check-list Manifesto! How would this work? Imagine creating a whole new system where one must register their research and the people conducting the work. This is sort of what happens when a clinical trial is begun or when a manufacturing plant runs a batch of drug production. A detailed process is developed over the course of considerable time, prior to beginning the actual work. The checklists here are detailed in quality management plans. The checklists are used to assure quality and to cover managements arses in dealing with regulatory agencies. We can do this too! The first box to check is to verify the players. As pointed out in the Bohannon experiment, the peers reviewing his paper did not even catch the fact that the authors listed and their institutions were fictitious. Checklist it!

Immediately we can see the difficulty in this undertaking. But since when has good science backed down from a challenge? This is the time that we must bend over backwards to make sure we are getting it right. Register your design of experiment. Register who is doing the work. Even if you are just going to verify the existence, quality or quantity of a protein via western blot, register that a western blot is going to be ran. In the QA box you must list how many times you ran the western blot and whether or not each one produces the same result.

Many people, especially the kind who become "peers" do not know the ways in which a western blot can be quality controlled. Manipulation of such images has been the cause of many a retraction. Using modern imaging technology however, you can have a time stamp and an original exposure setting that cannot be manipulated. That is quality. How do you control for that quality? Ask, in your checklist, how many western blots were run. Who ran the western. Do the western blots provide the same information from run to run, technician to technician, lab to lab... Using the density of a protein band and a known standard, what is the margin of error quantitatively? There are many such quality measurement that can be planned and incorporated into a checklist. At the end of such a quality control standardization, the checklist itself can be tested for its ability to maintain quality. Simply compare the new system to the peer review system.

There would be an outcry to going out and trying to fool the editors and peers working for Science, Nature or Cell. That is not what I am advocating here. Simply quantitate a protein in the western blot study I have outlined. Create the checklist, have several different labs run the westerns and generate a report, as outlined by the quality management plan, and be confident that your method is a reproducible and useful tool. Next, create a narrative where an elaborative RNAi experiment is to altar the quantity of the same protein tested in the western blot experiment. Don't fool the peer however. Just present them with a blinded experiment. They may receive real or faked western blot information. Let them use their peer review skills to pick out the real from the faked data.

There is a story out today about a two college professors who brought FDA advisory panels together with pharmaceutical representatives to "discuss" issues surrounding drug approval. This is similar to what we are talking about here today. Who gets a seat at the table? In this case big pharma was paying up to $35K to sit in on meetings where the FDA advisory panels fine tune their quality management plan. Why would big pharma pony up that amount of money to have a seat at the table? What would give these two college professors the audacity to even suggest the fee? These meetings still take place behind closed doors. The pharmaceutical companies just want a few more people to see and hear what is taking place. They are not paying for open access for all. Just a few more influencers whom they employ.

We, here at the CCS, believe that true open access and transparency is where science becomes useful. Patents, careerism, awards, financial gain all negatively effect the path of enlightenment. Peer review and pay-to-play secret FDA meetings are examples of poor quality management. We can do better.

John Bohannon and Science Are Afraid of Peer Review!

Derek Lowe has another interesting piece on a fun little experiment. It is glaring example of how bias gets in the way of everything. "Who's Afraid of Peer Review?" The author, Science, Derek Lowe and most commenters failed to acknowledge the similarity between the logic used in the fake paper and the logic of this experiment. First off, here is the experiment. Writer John Bohannon wrote a scientific manuscript akin to the Sokal Hoax. The author sent this manuscript to 304 publishers throughout the world. There were different names of authorship and fictitious institutions where they worked. The paper itself is full of easily spotted errors.

The papers describe a simple test of whether cancer cells grow more slowly in a test tube when treated with increasing concentrations of a molecule. In a second experiment, the cells were also treated with increasing doses of radiation to simulate cancer radiotherapy. The data are the same across papers, and so are the conclusions: The molecule is a powerful inhibitor of cancer cell growth, and it increases the sensitivity of cancer cells to radiotherapy.
There are numerous red flags in the papers, with the most obvious in the first data plot. The graph's caption claims that it shows a "dose-dependent" effect on cell growth—the paper's linchpin result—but the data clearly show the opposite. The molecule is tested across a staggering five orders of magnitude of concentrations, all the way down to picomolar levels. And yet, the effect on the cells is modest and identical at every concentration.
One glance at the paper's Materials & Methods section reveals the obvious explanation for this outlandish result. The molecule was dissolved in a buffer containing an unusually large amount of ethanol. The control group of cells should have been treated with the same buffer, but they were not. Thus, the molecule's observed "effect" on cell growth is nothing more than the well-known cytotoxic effect of alcohol.

That is a control error that I have seen purposely used many times in my biotech career. Don't get me started. I think the experimental design for this hoax is exactly what I have been talking about for years. It's not unlike developing a card trick. You first must understand your audience. Then you fool them. What Bohannon has done here is to design an oversimplified version of the tricks many biopharma people conduct daily. The more advanced a Cargo Cult becomes the more sophisticated this sort of in vitro card trick becomes.

What did Derek Lowe and most of his commenter miss? They too were being fooled. One commenter, Virgil, pointed out:

As is being Twittered and blogged elsewhere, there are some pretty HUGE problems with this, which are being missed by the mainstream scientific media in its reporting....
1) It's a news piece by a journalist, NOT a peer reviewed scientific paper.
2) There was no "control" group (i.e., non open access, non predatory journals). I would bet this would get into some higher up journals if only he'd tried.
3) There's no analysis whatsover - did the acceptance correlate with impact factor for example? What was the average review time? How many reviewers (just one, just editorial review, or more).
4) This is not about open access, that just happens to be in the title.
5) This borders on human subjects/outcomes research, but the author is not at a university, does not have IRB approval, did not get any kind of informed consent from the persons involved. As such, it's easy to see why this would be inadmissible as a research paper.
If you or I had done this and tried to get it published in Science, we would be crucified, but apparently if you're a journalist then the standards don't apply. So yeah, maybe if the headline was "hoity toity old media giant, feeling the pressure from open access, tried to pull one over on their new business rivals", then I'd be more inclined to believe it.

This is the most interesting part of the hoax. It was published in the journal Science under the title, "Who's Afraid of Peer Review?". The implication is that a superior peer reviewed journal, is to be feared by anyone trying to publish bad science. Open access does not offer any real resistance to bad science. Yet as another commenter points out:

I agree with Virgil. Note that this study has been published in Science WITHOUT a control experiment. The latter should have been done by submitting the bogus manuscript to the same number of non-OA journals. What is this telling us about the scientific competence of editors of high-impact journals?
I feel like submitting the same article signed by the most famous scientist in that field. Any bet on what percentage of respectable journals would accept it with minor corrections?

The control in the cell growth experiment lacked the same concentration of EtOH. In "Who's Afraid of Peer Review?" there is no control group at all. It appears that the editors of Science and John Bohannon were afraid to submit this paper, with a proper adjustment of author and institution, to a peer reviewed journal. The cargo cult connection here is not that the manuscript is Cargo Cult. True, it was dressed up like a real science manuscript and received a stamp of approval by most of the editors of the open access journals. The Cargo Cult connection is the the title of the article. "Who's Afraid of Peer Review?" Science and John Bohannon are!

The best analogy I can see is the Wesson Oil example in CCS:

The easiest way to explain this idea is to contrast it, for example, with advertising. Last night I heard that Wesson oil doesn't soak through food. Well, that's true. It's not dishonest; but the thing I'm talking about is not just a matter of not being dishonest; it's a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated at another temperature, they all will--including Wesson oil. So it's the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with.

Let's rewrite that with "Who's Afraid of Peer Review?"

The easiest way to explain this idea is to contrast it, for example, with advertising. Last night I heard that Open Access Journals publish bad science . Well, that's true. It's not dishonest; but the thing I'm talking about is not just a matter of not being dishonest; it's a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that All Journals Publish Bad Science, if it is submitted by certain scientists. If bad science is submitted by certain scientists (and caters to the popular beliefs of the peers), it all might be accepted--including Peer Review Journals. So it's the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with. 

The difference between Science and the open access journals put to the test here was not examined in this article. To be fair, it was just an article and not a peer reviewed scientific paper. Yet it was approved and published in the journal Science. No one at Science, nor those in on the hoax bothered to address the obvious missing control, that of testing superior peer reviewed journals.

"it's the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with"

I loved this experiment nonetheless. The point I make here today is that the experiment is only half over. The next step is to set out, as John Bohannon did, to purposely fool the subscription journals. Scientists are doing it everyday. Long ago Alan Sokal laid out the criteria for how this is done. Think like Bohanno or Sokal. Think like Diederek Stapel or Silvia Bulfone-Paus. The more we study ways to fool ourselves and the journals, the further we depart from Cargo Cult Science.

Merck Job Loss

The slaughter continues. At the same time small biotech companies are experiencing a blitzkrieg of IPOs, Merck is slashing another 8,500 jobs and ending R&D projects the management sees as fruitless. Merck will see savings of $2.5B per year from this move.

8,500 human beings will get their pink slip, put their cubicle decorations in a box and make that sad walk to the car. This on top of the already announced 7,500 job losses previously announced by Merck.

Man this blog is depressing! Sorry.

It brings to mind the many times I lost a biopharma job and what it meant to me and my career. Each job loss lowered my value. It doesn't matter that none of the companies I worked for are still in business except for the last. I established a history of 2 to 3 year stints at places no one has ever heard of. The people I worked with however were no different from one company to the next. We had good and bad, as is the case in any company. The bad ones that I've worked with were really bad. The good ones tended to be more adaptable to new situations, more open and honest and less antagonistic. There was a confidence that came from being good at what they do.

I do  this little thought experiment where I match up the people I've worked with. I have list, as everyone does, where I rank people who have worked in similar areas. For example, Pierre is the best protein purification specialist I have worked with and Hana is the worst. What would happen if those two had to work together at a company like Merck? Pierre had a PhD so he would be the boss over Hana in any biopharma company. Hana was absolutely clueless and like to show up for 5 hours a day. In this environment I assume that Hana would be let go and Pierre would establish an irreplaceable group of downstream process specialists. When Merck sets out to lop off 20% of the work force, Pierre and his people get to stay. But then, how do we know that Pierres superiors aren't more like Hana?

A real life experiment like this took place at my last job. We, a small biotech company, had an antibody drug but no manufacturing expertise. No one working at this small biotech company had ever seen the inside of a manufacturing plant where antibody drugs are made. We hired a CMO to take our master cell line and magically provide us with large quantities of our drug. For some reason, the small biotech had two upstream process development groups however. Neither group could establish a coherent, scalable, reproducible, method for growing our cells. One of the many problems was that the leader of group 1, Katie, would not share information with the leader of group 2. To remedy this situation we hired John, an upstream process development expert, from our CMO to lead a unified upstream process development team. He showed up and  began meeting his new team. Katie met one on one with John and went through three years worth of her work. I could see them through the glass wall of Johns office. Through the glass I could see Johns mood changing from glad, to sad, to mad in just a few hours of first meeting Katie. Within three days John had had enough. She had not produced one bit of upstream process knowledge that John could use. This was his new job and he had nothing to go on. John tried to fire Katie. Katie however was a favorite among the leadership. In a month John quit and Katie stayed.

Imagine now 16,000 people leaving Merck. How many Johns and Katies were there at Merck? How many Pierres and how many Hanas? It is not an option to assume that the CEO of Merck developed a scientific method to eliminate just the Katies and Hanas from his work force, sparing the Pierres. From my experience, not even a small biotech of roughly 40 people can sort out the wheat from the chaff. That is the problems with a Cargo Cult, when the planes don't come, whose fault is it?

It's our fault. We are not standardized, certified, highly qualified professionals. We are a random ensemble of college graduates inventing our jobs as we go along. How do you cure cancer when your boss gives you an antibody and tells you to get busy? There is no manual telling you what to do? Just a boss telling you what the data is suppose to support. Could we get the 16K former Merck employees together to write a manual on drug development? Could they write a book similar to the one that ex-Pfizer employees would write?

16,000 more biopharma experts are going back on the market. Who wants them? How does the next company determine the value of such individuals knowing that Merck turned them down? They will have a greater value than myself or my friend who just started a part time job at Home Depot after 20 years in biopharm. We come from places like Amgen, Gilead and a large assortment of 2 and 3 year jobs at places no one has ever heard of. We have worked with all kinds. Ex-Pfizer, ex-AstraZeneca... There is nothing special among us nor this new graduating class of 16,000 ex-Merck employees. They are all now just like us, just not as long term separated from the Cargo Cults.

So as to not end on a super depressing note, I will put it this way. This new group of 16,000 people are getting on a raft that will take them further down the river. It may take you to a better or a worse place. There are no guarantees. But it is the Katies and Hanas who most fear the unknown. They will stay right where they are for as long as they can. Get on the raft and go somewhere else. If I am correct in assuming that this is a Cargo Cult world, Merck will lose Katie/Hana/John/Pierre in equal numbers. It's all random. Mercks ineptitude will neither grow nor shrink. They'll just have less people to pay. Merck will thus have more dark days ahead. Get on that raft and go somewhere else.