The Counterfactual: Spreading fear and confusion with misleading formaldehyde studies

This article is from January 2015 but it just shows how they will use propaganda that has already been debunk over and over to mislead!

Another alarmist and deeply misleading story about formaldehyde and e-cigarettes has now emerged in the New England Journal of Medicine: Jensen et al Hidden Formaldehyde in E-cigarette Aerosols.

Short summary. This experiment, published in the New England Journal of Medicine, operated the vaping device at a such a high temperature that it produced thermal breakdown products (so-called dry puff conditions), but no user would ever be able to use it this way – the vapour would be too acrid.  They went on to calculate human cancer risk from these unrealistic machine measurements and presented the data in way that was bound to mislead, which it duly did and created a world-wide media storm. This was irresponsible science, careless publishing, and credulous journalism adversely changing the perceptions of the relative risks of smoking and vaping in a way that will cause harm. The paper should be retracted in its entirety.

Just before it was published,  I wrote to the corresponding authors and the editor of the journal, and I would like to share my open letters. 

Open letter to authors

I sent this letter to the authors 12 hours before the 5pm EST 21 January 2015 publication embargo, hoping they would have the integrity to pull it.

From: Clive Bates
Date: 21 January 2015 at 11:54 (GMT+2)
Subject: Hidden Formaldehyde in E-cigarette Aerosols – some questions and concerns
To: Peyton, Pankow

Hidden Formaldehyde in E-cigarette Aerosols – some questions and concerns

21 January 2015

Dear Dr Peyton, Dr Pankow

I write with reference to the forthcoming letter “Hidden Formaldehyde in E-cigarette Aerosols” to be published in the New England Journal of Medicine, under a 5pm EST 21 January 2015 embargo.

Given the great potential for these findings and the related cancer-risk calculations to cause damaging confusion and fear among smokers and vapers, I would be grateful if you could clarify the following:

1. What care was taken to ensure that the puff regime used was a reasonable proxy for human use and exposure? The letter does not detail any attempt to calibrate the puffing regime used in the experiment to match real-world vaping behaviour. The levels of formaldehyde detected suggest it was a highly unrealistic regime.

2. What, if any, precautions were taken to avoid measuring and reporting on ‘dry puff’ conditions – i.e. through use of such high voltage and high intensity puffing that the coil becomes so hot that it creates vapour of such acrid taste and harshness that human users would not use it in that way? It is under these conditions that high levels of formaldehyde and related compounds would be expected to form – but no human would ever be exposed to them. Humans have control over the sensory experience that puffing machines do not.

3. In making your newsworthy claims about cancer risk, what confidence do have that the puffing regime used appropriately represents human vaping behaviour, and therefore human cancer risk? There is a danger that naive reporting of your findings will characterise these risks as integral to vaping products, whereas they are a feature of the operating regime, which appears to be extreme in this case. These findings are only appropriate as cancer risk communication if the operating regime is realistic. However, the letter does not detail how you have assured this is the case – and no caveat has been provided to highlight that serious and probably fatal weakness in this work.

4. In the calculations of cancer risk, it is assumed that “inhaling formaldehyde-releasing agents carries the same risk per unit of formaldehyde as the risk associated with inhaling gaseous formaldehyde“. Can you provide a citation to support this assumption, given that the attention-grabbing findings in the letter rest entirely on it? As you will be aware, formaldehyde-releasing preservatives are used as an alternative to formaldehyde in many preparations for safety reasons.

5. The letter claims that the incremental lifetime cancer risk associated with long term vaping “is 5 times as high … or even 15 times as high … as the risk associated with long-term smoking“. Can you clarify that this comparison refers to only that part of the smoking cancer risk that arises from formaldehyde exposure? In order not to confuse readers with the idea that long term vaping may carry 5-15 times the risk of smoking, would it be possible to provide an appropriate context: for example, what proportion of the smoking cancer risk is attributable to formaldehyde? I think it is a small fraction of the total, and it would have been prudent to state this. Formaldehyde is not the most important carcinogen in cigarette smoke by some distance and just one of many. The Surgeon General’s 2010 report Chapter 5 provides a useful guide, but does not go as far as your letter does in attributing cancer risk to individual carcinogens. The Surgeon General also reminds us that:

Aldehydes such as formaldehyde and acetaldehyde occur widely in the human environment and are endogenous metabolites found in human blood

It is possible therefore that the estimation of cancer risk from formaldehyde is more complicated than your simple model allows for.

To be more direct, I am concerned that:

  • This study uses a completely unrealistic puffing regime to create the conditions in which formaldehyde forms with no attempt to calibrate the machine to reflect realistic human use.
  • That it presents results from extreme and unrealistic operating conditions which are then built into a ‘back of the envelope’ calculation of cancer risk.
  • That this contrived and artificial cancer risk is misleadingly compared to real human cancer risks associated with smoking.
  • That the statements about vaping having 5-15 times the incremental cancer risk associated with smoking are provided without context and could easily be misread as implying that vaping is more dangerous than smoking. It would not be the first time that misreporting of formaldehyde findings have created this impression.
  • This study may repeat the harm done through mischaracterisation of ‘light’ cigarettes by use of unrealistic puffing regimes that did not reflect real human behaviour. The same is likely to apply here, but instead of understating risk of a harmful product, the effect will be to grossly overstate the risk of a relatively benign product – with equally damaging results.

Many smokers have a great opportunity to switch from smoking to vaping, and to reduce their incremental risk of disease by 95-99%. However, studies like this and the reporting that has followed, are gradually persuading smokers that e-cigarettes are much more risky than they are, and that they might as well continue to smoke. A study published in 2014 found the following:

In 2010, 84.7% of smokers surveyed believed e-cigarettes were less harmful than traditional cigarettes, but according to this new study in 2013, that number dropped to just 65% [link].

This is a trend that should shame the public health community and the academics that are fuelling consumers’ misunderstanding with misleading studies that misrepresent risk.  I am sure it is not your aim to protect the cigarette trade and prolong the epidemic of smoking related disease, but it may well be the effect.

I hope you will take great care to ensure your findings are described in context and with appropriate caveats about whether these results are realistic for human exposure and that the calculations of cancer risk are remotely meaningful..

Yours sincerely

Clive Bates
London / Harare

[No competing interests]

New England Journal of Medicine had prior notice of the failures in this paper

After emailing the authors of the study with the question above, I also emailed the NEJM – more than eight before the embargo of 5pm EST on 21 January. At this point we only had the information in the letter (i.e. not including the equipment used), so I was pressing the NEJM to ask these question of the authors and in effect to pull the release and retract the letter if the answers were unsatisfactory.

From: Clive Bates
Date: 21 January 2015 at 15:50 (GMT+2)
Subject: Hidden Formaldehyde in E-cigarette Aerosols – some questions and concerns

Dear Editor NEJM
I am surprised that a prestigious publication like the NEJM has decided to publish the letter by Jensen et al describing their experiment with e-cigarettes and related calculation of cancer risks. The most basic disciplines appear not have been followed, and it is likely that the test results were obtained by operating the device under conditions that no human vaper would be willing or able to tolerate.  The danger is that misleading studies like like will scare people about e-cigarettes for no reason and encourage more smokers to stick with smoking – in this field especially there is a great responsibility to conduct realistic experiments, report results and caveats fairly, and not to indulge in simplistic assertions of cancer risk.
I have written a letter to the authors to put these issues to them.  This is below.  May I suggest that NEJM editors review this letter and its peer reviews with a view to deciding if this paper provides a useful contribution or a harmfully misleading contribution to the literature.
Clive Bates
[letter to authors as above attached]

Naivety of these authors

These two authors wrote about e-cigarettes in November 2014 (Chemists break down e-cigarette research) and in doing so demonstrated that they are basically clueless:

Peyton agreed with Pankow. He also pointed out that the high temperatures to which the element heated e-cigarette additives—over 600 degrees celsius—resulted in the creation of molecules not previously seen.

Vaping is typically done at 200-260 degrees celsius, with dry puff conditions developing at around 280 degrees.

Commentary and analysis on this study

Dr Konstantinos Farsalinos: The deception of measuring formaldehyde in e-cigarette aerosol: the difference between laboratory measurements and true exposure

The scientific community must realize that variable wattage devices cannot be used at any wattage levels with any available atomizer. Even for naïve users, the harsh taste of the dry puff phenomenon is unbearable. … In fact, it is very easy to produce as much aldehydes as you want in the lab with an e-cigarette device. However, this has nothing to do with exposure from e-cigarette use.

Dr Michael Siegel:  New Study Reports High Levels of Formaldehyde in Electronic Cigarette Aerosols

Essentially, what this study demonstrates is that if you overheat a vaping system, it will produce high levels of formaldehyde. However, such conditions are not realistic, as they could not be tolerated by an actual vaper. Therefore, extrapolating from this study to a lifetime of vaping is meaningless.

Norbert Zillatron: Freaking Formaldehyde

…you can’t just simply select an arbitrary puffing regime and expect it to represent an applicable model of real vaping

Tom Pruen (ECITA) First burn the barrel, then scrape it – commentary on a letter published in the NEJM

Consumers are extremely unlikely to voluntarily inhale high concentrations of formaldehyde; formaldehyde is characterized by its unpleasant smell, and at concentrations of as little as 5 parts per million causes burning sensations in the respiratory tract, and breathing difficulty

Professor Peter Hajek: Formaldehyde in e-cigarettes: expert responds

When a chicken is burned, the resulting black crisp will contain carcinogens but that does not mean that chicken are carcinogenic. Without overheating the e-liquid, no formaldehyde was detected.  Vaping may not be as safe as breathing clear mountain air, but it is much safer than smoking. It would be a shame if this study persuaded smokers who cannot or do not want to stop smoking and contemplate vaping that they might as well stick to their deadly cigarettes.

Professor Brad Rodu: Bogus research of formaldehyde in e-cig vapor

R. Paul Jensen and colleagues at Portland State University produced the new results by overheating an e-cigarette, a condition (called dry puffing) that is familiar to vapers; the resulting product tastes so bad it cannot be inhaled. In other words, the formaldehyde produced under abusive conditions is not “hidden” at all, because it is in vapor that users find intolerable.  Enough data is extracted from the authors to confirm the the measurements were made in extreme and unrealistic ‘dry puff’ conditions. Konstantinos Farsalinos has the science: Verified: formaldehyde levels found in the NEJM study were associated with dry puff conditions. An update.

Dr Konstantinos Farsalinos figures out what they actually did: Verified: formaldehyde levels found in the NEJM study were associated with dry puff conditions. An update.

It is more than obvious that the findings of very high levels of formaldehyde are a result of overheating. Lack of experience on e-cigarettes and no contact with vapers can result in such erroneous and unrealistic results, which can create confusion and misinformation both in the scientific community and among users and potential users of e-cigarettes.

Dr Michael Siegal follows up: Confirmed: Formaldehyde Study Conducted Under Implausible Conditions; Conclusions Invalid

Unfortunately, the alarmist (and incorrect) conclusions of this study have already been widely disseminated in the media. Even if the information is corrected, it appears that the damage is done.  I believe that the damage is substantial because many smokers will now become convinced that there is no advantage to switching from tobacco cigarettes to electronic cigarettes.

Dr Gilbert Ross of the American Council on Science and Health, Poorly-done e-cig vapor study gets big headlines but means nothing. Dr Ross highlights the additional point that it wasn’t formaldehyde that was measured and formaldehyde is not an especially powerful carcinogen.

This flawed study will be used to attack e-cigs as not only not safer than smoking cigarettes, but perhaps even more toxic. Nothing could be further from the truth. Those who promulgate this falsehood should be aware that even if the study were done correctly — which it was not — such a conclusion is the worst kind of destructive fiction. Formaldehyde was not even studied, in fact, but something called formaldehyde releasing compounds. And formaldehyde is a very weak carcinogen, with only a slightly increased chance of cancer among highly-exposed workers over a lifetime. The presence of one such carcinogen in vapor cannot be compared to the toxic stew of carcinogens and other toxins in cigarette smoke.

A small number of journalists show they are sufficiently independently minded to challenge this alarmist drivel.

And it got the satirical treatment it deserved on Gutfeld: New e-cig study goes up in smoke “The point: If you create an unrealistic climate for harm you create the harm you seek, which allows researchers to elevate the risk and that lands them in a prestigious journal.”

Sample press coverage

Without exception the press coverage refers to elevated cancer risk.


This study is so poorly designed and inherently misleading that it should not have been conducted, should not have been written up, should not have been published in the NEJM or anywhere, should not have been pitched to the media, and its findings should be completely ignored as worthless. The authors should retract the letter.  If they are unwilling to do that, the NEJM should retract it.

Post script: Infantile response from tobacco control

Oh but wait… it was well received and acclaimed by some in the high command of tobacco control.

Screen Shot 2015-01-22 at 07.00.34

Technical note: dry puff phenomenon

Professional researchers, peer reviewers and editors designing and publishing experiments on vaping devices should, as in any field, acquaint themselves with the peer-reviewed literature already published.  This would help them avoid designing hopelessly flawed experiments and rushing into print with misleading results and bogus estimates of cancer risk.  If Jensen et al had applied this most elementary discipline, they would have found this description of the dry puff phenomenon and its implications for experimental protocols (Farsalinos et al 2013):

3.4.1. Vaping vs. Smoking Topography

Vaping topography may have significant implications in production and delivery of potentially harmful substances. The EC evaporation rate and thermal load are directly dependent on the puff duration and interpuff interval. If the device is activated before the temperature is significantly decreased and/or before the wick is sufficiently supplied with liquid, the device will get overheated. This causes a phenomenon called “dry puff”. It is an unpleasant, burning taste that forces the user to lower puff duration and increase interpuff interval. It is also reproduced when the atomizer has very low amounts of liquid, signalling that it should be refilled. This phenomenon occurred in some experienced users when they were asked to use the “eGo-C” atomizer in this study. They had to lower puff duration and interpuff interval in order to avoid “dry puff”, while no such problems occurred with the “Epsilon” atomizer. Although not tested yet, there is a theoretical concern that overheating the EC may lead to production of significant amounts of toxic substances like acrolein or formaldehyde, which can be formed from thermal degradation of glycerol in a closed chamber [21,22]. The “dry-puff” phenomenon, although easily detected and avoided by the user, cannot be detected in the laboratory setting. Therefore, if this occurs during a laboratory experiment, it will significantly undermine the value of the study results and their applicability to real use. It should be emphasized that each type of atomizer has different cooling and liquid-supply abilities, depending on the design and material used. This should be taken into consideration when preparing laboratory research protocols.

Farsalinos KE, Romagna G, Tsiapras D, Kyrzopoulos S, Voudris V. Evaluation of electronic cigarette use (vaping) topography and estimation of liquid consumption: implications for research protocol standards definition and for public health authorities’ regulation. Int J Environ Res Public Health 2013; 10: 2500–14.

Notice the title: implications for research protocol standards definition. The issue was made even more explicit in the following, again with Farsalinos leading, with the obvious suggestion that a human vaper is asked to validate the machine puffing protocol:

An important issue that needs to be clarified before proceeding with laboratory experiments is the determination of the “dry puff” phenomenon [11]. It occurs when insufficient liquid is supplied to the wick of the atomiser, leading to temperature elevation. This is detected by the user as an unpleasant burning taste which is avoided by reducing puff duration and increasing interpuff interval. Therefore, if this phenomenon is reproduced in the laboratory setting it does not represent EC use in realistic conditions. Since no laboratory method has been developed to detect it, one of the researchers (who is an experienced EC user) was assigned to test both devices in order to detect the dry puff phenomenon.”

Farsalinos KE, Romagna G, Allifranchini E, Ripamonti E, Bocchietto E, Todeschi S, Tsiapras D, Kyrzopoulos S, Voudris V. Comparison of the cytotoxic potential of cigarette smoke and electronic cigarette vapour extract on cultured myocardial cells. Int J Environ Res Public Health. 2013 Oct 16;10(10):5146-62.

The Japanese Formaldehyde Fiasco revisited

Also, it is worth mentioning that this is not the first time an overhyped formaldehyde story has made its way into print rather than the bin. Excellent blogs by Konstantinos Farsalinos [Electronic cigarette aerosol contains 6 times LESS formaldehyde than tobacco cigarette smoke] and Brad Rodu [Formaldehype vs. Fact: Levels Are Far Lower in E-Cigarettes Than In Cigarettes] give the full story of the Japanese rogue result that made headlines worldwide.  At the time, I also wrote to the author of this study, Dr. Naoki Kunugita, to point out the irresponsibility of his statements. Here is the letter from November 2014.

Date: 29 November 2014 at 14:31
To: “Dr. Naoki Kunugita M.D. Dr.Med.Sci.” < >
Dear Dr Kunugita

I hope you are aware of the impact that your comments on e-cigarette formaldehyde exposure has caused in the press, and therefore in public perception of risk of e-cigarettes relative to smoking. You can read some of it here:

The impression that has been left is exactly the opposite of a realistic appraisal of exposure to carcinogens in general and formaldehyde in particular that a responsible authority would wish to communicate to the public.  Your remarks appear to be based on a single unpublished measurement: “In one brand of e-cigarette the team found more than 10 times the level of carcinogens contained in one regular cigarette” as quoted by AFP.   Can you provide the data that supports this argument? I have been unable to locate it in the published literature.

A more realistic appraisal across the range of measurements would suggest formaldehyde exposures far lower than for smoking – at least six times lower based on your own data and perhaps fifty times lower would be an appropriate characterisation.  I hope you have seen the critique of your work and communication by Dr Konstantinos Farsalinos (here)

However, you have chosen to communicate an extreme result that is not open to scrutiny. Because it is not published or replicable, we cannot know for sure: but is likely that this device was running very hot and/or dry to generate emissions like this. No human user would continue with this mode of operation as the taste would provide immediate feedback to behaviour (something that does not happen with machines).  It follows that the result (even if accurate) you have communicated to the public is:

1. an artefact of the machine testing regime and device settings;
2. not a realistic risk for human exposure.

Whether or not it was the aim of your communication, the effect has been to cast doubt in the minds of many smokers about the benefits of switching to e-cigarettes, which would be immensely positive to their health.  The perception of risk arising from e-cigarette use is already hugely exaggerated by the public and this will make the misalignment of perception and reality even it worse.

It is not sufficient to argue that your statement was technically correct (yes, you may have a result like this). It is essential that when respected institutions and experts make comments they take care to ensure the effect it has on perception is balanced and proportional, does not spread false alarm and helps people understand the risks rather than mislead them.  You have a responsibility to be both truthful and to ensure you work is placed properly in context.

Given this has not happened with this statement, I am writing to ask you to make a proper balancing statement that would put this in context and restore some reality to the discussion of e-cigarette risks.

I hope to hear from you soon and that you will take this request seriously – I am sure you do not want to be responsible for supporting continued cigarette use by confusing people about e-cigarette risk.

Yours sincerely

Clive Bates

Disclosure: I am a long standing supporter of harm reduction techniques for reducing tobacco related disease. I have no competing interests.   For further reading please see my publication: WHO Position on ENDS (e-cigarettes): a critique of the use of science and risk communication

Of course, as with many academics and bureaucrats, he felt no compulsion to reply or to be in any way accountable for the sorry mess he created.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s