Creative thinking: trying to find methodology problems when we don’t like the results of research
- Friday, 28 November 2014 18:13
Creative thinking: trying to find methodology problems when we don’t like the results of research
By Dr Farsalinos
I was just informed about a comment posted by Prof Glantz on his blog, concerning the study: “Impact of Flavour Variability on Electronic Cigarette Use Experience: An Internet Survey” published in International Journal of Environmental Research and Public Health on December 2013. The study evaluated the patterns of flavours use and their impact on pleasure, satisfaction and smoking reduction or cessation in a group of dedicated e-cigarette users (more than 4,000 participants).
Prof Glantz discovered a problem in the methodology, saying that: “The problem with this study is that the sample was recruited from the e-cigarette advocacy site www.ecigarette-research.com, which is hardly a random sample of e-cigarette users or potential users (including kids)”.
First of all, the study recruited participants from major e-cigarette forums and websites. Obviously, we wanted to assess the experience of e-cigarette users, and that was the best way to recruit as many vapers as possible. Secondly, there is no such thing as a random sample because, in every case of surveys, participants are those who accept to participate, unless someone has a way to force a random sample to participate irrespective of their will. Thirdly, I am certain that less than 1% of e-cigarette consumers use flavourless liquids and I supose Prof Glantz has never met a single vaper using unflavoured liquid. Finally, it seems that Prof Glantz is “breaking through an open door”. In the study results, we specifically mentioned: “There are some limitations applicable to this study. The survey was announced and promoted in popular EC websites. Therefore, it is expected that dedicated users with positive experience with ECs would mainly participate, and the high proportion of former smokers confirms this. However, it is important to evaluate the patterns of use in smokers who have successfully quit smoking, since this can provide health officials with information on how to educate smokers into using ECs, especially during the initial period of use”.
I think the last sentence is creating the most problems to those with a pre-determined ideology against e-cigarettes. They do not want us to explore why e-cigarettes are successful in substituting smoking. Undoubtedly, dedicated vapers are the most successful users, with most of them being heavy ex-smokers who managed to quit smoking through e-cigarette use. It is of outmost importance to explore why and how these people got rid of tobacco cigarettes. Therefore, I consider the selection of participants as a strong point (rather than a limitation) of that study, and this is exactly the population group we wanted to assess. Such a selection would create problems only if we wanted to assess smoking cessation rates. However, in no such survey did we ever support (or even imply) that the success rate of e-cigarettes in smoking cessation could be examined. On the contrary, we specifically mentioned in our worldwide survey: “It should be emphasized that participants in these surveys are mostly dedicated users... The 81% of participants reporting complete smoking substitution cannot be interpreted as the true potential of ECs in smoking cessation in the general population; controlled studies have found much lower cessation rates”.
Our methodologies and approach of clearly presenting all information in its true context is in contrast to what other groups have done. For example, Grana et al. published a longitudinal analysis of e-cigarette use and smoking cessation in JAMA Internal Medicine. They appear to have recruited a national sample of US smokers: “We conducted a longitudinal analysis of a national sample of current US smokers to determine whether ecigarette use predicted successful quitting or reduced cigarette consumption”. The trick was that the comparison was between e-cigarette users and non-users based on BASELINE use. So, the “representative sample” was e-cigarette users who had failed to quit smoking at baseline (because they were all current smokers)! No surprise, they found that e-cigarette use did not predict quitting at 1 year.
In conclusion, in every study we have published, we made sure that the methodology was described clearly, the results were presented in detail, and the interpretation was made with strictly scientific criteria.
