INTRODUCTION

Effective tobacco control policies help denormalize smoking, decrease smoking prevalence1, and reduce morbimortality attributable to tobacco2. Many efforts have been made globally to tackle the tobacco epidemic3, stimulated by the enforcement of the Framework Convention on Tobacco Control. In the European Union (EU), the Tobacco Products Directive has driven the application of stringent tobacco control policies to reduce tobacco use and its negative consequences on health. However, the implementation and enforcement of tobacco control policies still vary greatly across Europe4.

Among the initiatives to monitor the implementation of tobacco control policies in Europe, Joossens and Raw5 developed the Tobacco Control Scale (TCS) in 2006. The TCS score is determined by a questionnaire based on six cost-effective policy interventions that should be prioritized according to the World Bank. These measures include taxation, smoke-free policies, public spending in information campaigns, advertising bans, health warnings, and treatment. The score assigned to each of these components is weighted by its reported evidence-based effectiveness. Therefore, the score attributed to each country increases with the strength of tobacco control policies up to a maximum of 100 points, indicating full implementation5.

At its inception, the aim of the TCS was to monitor the progress in tobacco control in Europe at a national level by comparing the performance of countries by their ranking5 and to inform the agenda by highlighting the policy components for which progress is lacking, as well as the countries or regions most affected by such delays6. Since 2006, the TCS has been updated every three years (available at www.tobaccocontrolscale.org).

Evaluating the impact of tobacco control policies among the population has become an important research area; thus, the TCS has been used as a research tool to measure the implementation of tobacco control policies, though it was not designed for such purposes. However, little is known about the use of the TCS by the tobacco control research community and its advantages and disadvantages as a research tool. Therefore, our aim was to characterize the use of the TCS by researchers and its main limitations and strengths as a research tool in order to critically assess its use as a research instrument.

METHODS

Data sources

We performed an extensive literature search in the online databases PubMed and Web of Science to identify publications that have used the TCS score(s) as an independent or dependent variable from 27 March 2006, when the first TCS was published, until 1 December 2019. The search was conducted using ‘tobacco control scale’ as the keyword in all fields without any other restrictions to ensure a very sensitive search. The Ethics and Clinical Research Committee of the Hospital Universitari de Bellvitge approved this study (PR247/18).

Study selection

We identified 69 publications (32 duplicated in both databases). After removing duplicates, two researchers (AF and AB) screened the titles and abstracts, obtaining 32 studies. The inclusion criteria were quantitative research and inclusion of the TCS score(s) (as dependent or independent variable) in the analysis. We found 27 eligible publications (Figure 1). We completed our search by manually reviewing the reference lists of the selected papers and by conducting the same search in Google Scholar (www.scholar.google.com; with search terms in English). These additional searches provided five new publications that met the inclusion criteria and the full-texts were reviewed.

Figure 1

Flowchart of the selection process of publications for full-text review

https://www.tobaccoinduceddiseases.org/f/fulltexts/128318/TID-18-91-g001_min.jpg

Therefore, we finally included 32 publications that used the TCS score(s) as a dependent or independent variable.

Data extraction

A detailed protocol and Microsoft Access® database were designed to extract and register the information from each publication. The evaluation protocol was developed by three researchers who are experts in tobacco control (AF, CM, and EF). The protocol describing the main objectives, information sources, search strategy and eligibility criteria, and the data collection was reviewed and approved by all researchers. All variables for which data were described in the publications’ Methods sections were listed.

Two reviewers independently extracted the data according to the protocol (AF and AB). If any discrepancies emerged, the reviewers discussed the papers until agreement was reached and, when no consensus was met, divergences were solved by discussing them with a third reviewer (CM). The evaluation process was conducted in January 2020.

The extracted information included general characteristics, publication characteristics, research characteristics, and characteristics of the use of the TCS (Table 1).

Table 1

Summary of the information extracted (variables and categories) from each publication

VariablesCategories
GeneralAuthor surname and initials, institutional affiliation, and country of affiliation of the first author
FundingYes, no, or not declared; and if yes, private, public, or both
Conflicts of interestYes, no, or not declared
PublicationType of publicationResearch paper, brief paper, review, letter to the editor, editorial, comment, or other
Publication year
Journal
Open AccessYes or no
ResearchObjective of the study
Type of designObservational or experimental; cross-sectional or longitudinal
Type of study dataIndividual, ecological, or multilevel, with individual data from surveys as first level unit with TCS score by country as second level of aggregated data
Main results( literals )
Limitations (specifically, those related to the use of the TCS as a tool to monitor tobacco control policy implementation)( literals )
Main conclusions and their purposeFor advocacy (when directly addressed to stakeholders and policymakers), to undertake further research on the topic, or both
Use of the Tobacco Control Scale (TCS)Type of variableDependent, independent, or both
Year of the TCS report
Source of the TCS score(s)Primary source, when publications included the TCS score(s) from the original reports; or secondary source, when publications included TCS data from other publications in which case the alternative data source was recorded
Total scoreYes or no
Individual components score(s)Yes or no; and if yes, which components were included
Countries from the original TCS report(s)Yes or no; and if not, we recorded the number of countries included

RESULTS

The 32 publications were published between 2008 and 2019. More than two-thirds (n=23; 71.9%) were published by research groups from the Netherlands (n=8), Spain (n=9), and the United Kingdom (n=6). Almost all (n=30; 93.8%) were original articles published in peer-reviewed journals, 12 of which were Open Access (37.5%). In addition, almost all declared no conflicts of interest (n=29; 90.6%) and were financially supported (n=26; 81.3%) with public funds (n=19; 73.1%), both public and private funds (n=5; 19.2%), or private funds (n=2; 7.7%).

Most of the publications (n=31) were observational studies; 16 used ecological data (50%) with the country as the unit of analysis and 16 used multilevel data (50%) with individual data from surveys as the first-level unit and TCS score by country as second-level aggregated information. Regarding the study design, 23 of the publications were crosssectional studies (78.6%). Most of the articles (n=24; 75%) (Table 2) included the TCS score(s) as an independent variable from primary reports, whereas 10 publications (31.3%) (Table 3) used the scores from secondary sources that calculated a new score based on the TCS methodology. Overall, 87.5% (n=28) of the publications used the overall TCS score by country and 65.6% (n=21) used the individual policy component scores.

Table 2

Main characteristics of the studies that used the original TCS reports as primary data sources (N=22)

Reference, locationObjectiveStudy designUse of TCS (Type of variable, original data, total and components score)Main resultsLimitationsConclusions
Feliu et al.8 2019, SpainTo empirically evaluate whether the hardening hypothesis can be confirmed at the population level in the 28 EU member states, and to analyze the determinants of hardcore and light smoking considering both individual and contextual country-level characteristicsObservational, multilevel, time-trendsIndependent variable, original data, only total scoreHardening smoking is not increasing in EU member states where smoking prevalence is decreasing. Odds of being a hard-core smoker are higher among middle-aged men of lower class and lower in countries with higher TCS scoresNo limitations reported about the TCS as a variableAdvocacy
González-Marrón et al.13 2019, SpainTo explore the association between the implementation of tobacco control policies and the risk of lung cancer in the EUObservational, multilevel, cross-sectionalIndependent variable, original data, only total scoreSignificant inverse correlation between TCS 2010 and the proportion of former and ever smokers at high risk of lung cancer according to NELSON criteriaThe cross-sectional design of TCS limits the validity of the study to establish causationAdvocacy
and research
Feliu et al.4 2019, SpainTo assess the midterm association of tobacco control policies on smoking prevalence and quit ratios among 27 EU countriesObservational ecological, and cross-sectionalIndependent variable, original data, total score and by componentsIn EU27, countries with higher scores in the TCS has lower prevalence of smokers, higher quit ratios, and higher relative decreases in their prevalence of smokers over the last decadeThe TCS does not score the level of enforcement except for smoke-free policies and the score may not fully reflect tobacco control policies implemented in subsequent years. The ranking of countries according to TCS scores has been relatively consistent across different editionsAdvocacy and research
Diez-Izquierdo et al.7 2018, SpainTo evaluate the correlation between tobacco control policies and the prevalence of preterm births and low birthweight in the European countriesObservational,ecological, and cross-sectionalIndependent variable, original data, total score and by componentsThe TCS score negatively correlated with the prevalence of preterm births for <37 weeks and <32 weeks and the prevalence of low birthweight (<2500 g) in European countries in 2010No limitations reported about the TCS as a variableAdvocacy
Lidón-Moyano et al.18 2018, SpainTo describe the acceptability of some tobacco product regulations and to explore their relationship with tobacco control legislation levels in EuropeObservational,ecological, and cross-sectionalDependent and independent variable, original data, total score and by componentsStrong support for tobacco product regulations was observed. A positive relationship was found between TCS scores and support for tobacco product regulations at both the ecological and individual levelNo limitations reported about the TCS as a variableResearch
Filippidis et al.22 2017, UKTo examine associations between median cigarette prices, price differentials between cigarette brands, and infant mortalityObservational, ecological, and longitudinalIndependent variable, original data, only by components scoreLarger differences between median and minimum cigarette prices were associated with increased rates of infant mortality. Median price increases during the study period were associated with 9208 fewer infant deaths, but a further 3195 infant deaths could have been avoided if no price difference was observed between minimumpriced and median-priced cigarettesNo limitations reported about the TCS as a variableAdvocacy
Filippidis et al.24 2017, UKTo explore factors associated with self-reported exposure of the EU population to tobacco products and electronic cigarette advertisingObservational, multilevel, and cross-sectionalIndependent variable, original data, only by components score40.0% and 41.5% of respondents reported seeing any e-cigarette and tobacco product advertisement within the past year. Smokers, males, younger respondents, those with financial difficulties, people who had tried e-cigarettes, and daily internet users were more likely to report it. Respondents in countries with more comprehensive advertising bans were less likely to selfreport exposure to any tobacco, but not to e-cigarette advertisementsNo limitations reported about the TCS as a variableResearch
Kuipers et al.23 2017, the NetherlandsTo estimate the impact of introducing sales restriction laws in European settings and to test whether the impact of the laws differed between adolescents of high and low socioeconomic positionExperimental, multilevel, and cross-sectionalIndependent variable, original data, total score by componentsNo decrease in smoking in countries that introduced sales restrictions for minors (2007–2011) compared to countries that introduced these restrictions earlier (before 2007). Sales restrictions were associated with a stronger decrease in perceived ease of the obtainability of cigarettes. The results were similar for adolescents of high and low socio-economic positionNo limitations reported about the TCS as a variableAdvocacy
Lidón-Moyano et al.12 2017, SpainTo analyze the correlation between the implementation of tobacco control policies and tobacco consumption, particularly rolling tobacco and e-cigarettes, and the intent to quit smoking in 27 countries of the EUObservational, ecological, and cross-sectionalIndependent variable, original data, total score and by componentsAn inverse correlation between TCS score and prevalence of smoking of conventional cigarettes and a positive correlation with the intent to quit smoking within the past 12 months. The correlation between TCS and secondhand smoke (SHS) exposure at work was negative. Significant negative correlation between TCS score and the prevalence of having ever tried a waterpipeThe 2-year gap between the measure of the TCS and the Eurobarometer survey does not allow detection of the effect of measures adopted between 2010 and 2012Advocacy
Allen et al.35 2016, UKTo evaluate the potential effectiveness of maximizing the TCS score for the UK using a model stratified by socio-economic circumstances and to illustrate health improvements associated with reduced smoking prevalenceObservational, ecological, and longitudinalIndependent variable, not original data, only total scoreImprovements in tobacco control policies towards maximum TCS score could substantially reduce smoking prevalence and reduce health-related inequalitiesImplementation was not consideredAdvocacy
Ferketich et al.1 2016, USATo determine the relationship between the TCS score and the prevalence of in-home smoking bans and beliefs on other tobacco control policiesObservational, multilevel, and cross-sectionalIndependent variable, original data, total score and by componentsThe TCS score was correlated with the prevalence of inhome smoking bans. Four of the individual contributions to the TCS scale (price, public campaigns, smoking bans and health warnings) were significantly related to inhome smoking bansNo limitations involving the use of TCSAdvocacy and research
Pförtner et al.10 2016, GermanyTo address to what extent different measures of the TCS are associated with smoking in adolescence in 29 European countries and how the association between tobacco control policies and smoking varies by family affluenceObservational, multilevel, and cross-sectionalIndependent variable, original data, only by components scoreTobacco control policies did not strongly interact with Family Affluence Scale (FAS) predicting adolescent smoking. For boys, prevalence of smoking decreased with higher tobacco price regardless of the socio-economic background. For girls, no difference was found in smoking prevalence by FASa. The limited number of observations at country level and the low variance of some tobacco policies across countries may reduce the robustness of parameter estimates; b. Analyzed data from the TCS has changed since 2006Advocacy and research
Filippidis et al.15 2016, UKTo explore whether exposure to SHS among non-smokers in the EU showed any association with sociodemographic factors and/or the extent of national tobacco control policiesObservational, multilevel, and cross-sectionalIndependent variable, original data, total score and by components29.0% of non-smoking participants reported being exposed to SHS in indoor areas. Males and individuals with difficulties paying bills had significantly greater odds of being exposed. For every unit increase of a country’s score on the Smoke-free Component of the TCS, the probability of reporting exposure to SHS increased in bars, restaurants, and workplacesNo limitations reported about the TCS as a variableAdvocacy
Martínez- Sánchez et al.14 2014, SpainTo evaluate the correlation between the implementation of tobacco control policies and smoking prevalence in private venues in the 27 countries of the EUObservational, ecological, and cross-sectionalIndependent variable, original data, total score and by componentsNo correlation was found between the implementation of smoke-free legislation at work and in public places and an increase in prevalence of smoking in private venues in the EU. More developed smoke-free policies positively correlated with a high prevalence of smoke-free housesNo limitations reported about the TCS as a variableAdvocacy and research
Rughinis et al.19 2014, RomaniaTo investigate the relationship between the number of cigarettes smoked daily and habits and beliefs concerning passive smokingObservational, multilevel, and cross-sectionalIndependent variable, original data, only total scoreLight smokers are less likely to have houses and cars in which smoking is allowed, to have visited drinking or eating establishments that allow smoking, and to be systematically exposed to tobacco smoke in the workplaceNo limitations reported about the TCS as a variableAdvocacy
Kovess et al.16 2013, FranceTo ascertain patterns of parental smoking in the vicinity of children in Eastern and Western Europe and their relation to TCS scoresObservational, multilevel, and cross-sectionalIndependent variable, original data, only total scoreEastern European parents were about twice as likely to smoke near their children as those in Western Europe. Current maternal smoking prevalence was similar. A strong relationship was observed between parental education, tobacco control policies, and smoking near the childNo limitations reported about the TCS as a variableAdvocacy
Willemsen et al.20 2012, the NetherlandsTo examine how a country’s level of tobacco control is associated with markers of denormalization of smoking, smoking prevalence, and societal support for tobacco controlObservational, ecological, and cross-sectionalDependent and independent variable, original data, only total scoreSmokers in EU countries with higher TCS scores are more concerned about the effect of smoking. Support for tobacco policies is higher in countries with more concerned smokersNo limitations reported about the TCS as a variableAdvocacy and research
Bogdanovica et al.25 2011, UKTo test the hypothesis that smoking prevalence is higher in countries with high levels of public sector corruption and explore the ecological association between smoking prevalence and a range of other national characteristics in current EU Member StatesObservational, ecological, and cross-sectionalDependent variable, original data, total score and by componentsSmoking prevalence was significantly higher in countries with higher scores for corruption, material deprivation, and gender inequality, and lower in countries with higher gross domestic product per capita, social spending, life satisfaction, and human development scoresNo limitations reported about the TCS as a variableAdvocacy
Martínez- Sánchez et al.21 2010, SpainTo describe the correlation between the TCS and smoking prevalence, self-reported exposure to SHS, and attitudes towards smoking restrictions in the 27 countries of the EUObservational, ecological, and cross-sectionalIndependent variable, original data, total score and by componentsA direct non-significant association was found between TCS scores and the prevalence of smoke-free houses, and a non-significant inverse correlation with allowing smoking in certain roomsNo limitations reported about the TCS as a variableAdvocacy
Tual et al.17 2010, FranceTo explore the relationship between SHS exposure and the strength of national-level tobacco control policiesObservational, multilevel, and cross-sectionalDependent variable, original, data only total scoreThe Carbone monoxide concentration decreased with the strength of tobacco control policies, as scored by the TCS in a large non-smoker European populationNo limitations reported about the TCS as a variableAdvocacy and research
Hublet et al.9 2009, BelgiumTo investigate smoking policies in 29 European countries in relation to the national smoking prevalence among young peopleObservational, multilevel, and cross-sectionalIndependent variable, original data, total score and by components3.8% variance in regular smoking in boys and 3.5% in girls can be attributed to country structure or country of residence. In boys, this variance is associated with country-level tobacco controlNo limitations reported about the TCS as a variableAdvocacy
Schaap et al.11 2008, the NetherlandsTo examine the extent to which tobacco control policies correlate with smoking cessationObservational, ecological, and cross-sectionalIndependent variable, original data, total score and by componentsHigh-educated smokers were more likely to quit smoking in all age-sex groups. TCS score was positively associated with quit ratios in all age-sex groups, with no consistent differences between high and low educationThe information described by the TCS score is about policies in 2005 and recently implemented policies are not incorporated; therefore, the impact of such policies may be underestimated when using the current version of the TCSAdvocacy
Table 3

Main characteristics of the studies that used the TCS from a secondary source and estimated the scores of the countries using TCS methodology (N=10)

Reference, locationObjectiveStudy designUse of TCS (Type of variable, original data, total score and/or components)Main resultsLimitationsConclusions
So et al.26 2019, UKTo describe changes in smoking prevalence over time within EU member states from 2009–2017; to describe how within-country and between-country variations in the implementation of tobacco control policies are associated with current smoking in individuals; and to describe how these variations affect individuals of different socioeconomic positions.Observational, multilevel, and longitudinalIndependent variable, not original data, only total scoreA general trend of decreasing smoking prevalence over the last decade was found in the EU. There was significant variation at the country level and country-year level, indicating that countries differed significantly in their smoking prevalence trajectory. Strong tobacco control policies were significantly associated with lower odds of being a current smoker, with a greater effect in upper class occupationsNo limitations reported about the TCS as a variableResearch
Serrano- Alarcón et al.27 2019, PortugalTo evaluate the impact of tobacco control policies on smoking among older adults in Europe from 2004–2013Observational, multilevel, and longitudinalIndependent variable, not original data, total score and by componentsA 10-point increase in the TCS score was associated with a drop in the probability of smoking by 1.1 percentage points (not significant). Pricing and smoke-free policies were significantly associated with smokingNo limitations involving the use of TCSResearch
Bosdriesz et al.31 2016, the NetherlandsTo assess whether tobacco control policy was associated with socioeconomic inequalities in smoking across the EU in the period 2006– 2012Observational, multilevel, and longitudinalIndependent variable, not original data, total score and by componentsAn association was found between tobacco control policies and smoking cessation among higher educated smokers. In middle- and high-educated smokers, policies were also associated with a decrease in smoking intensityNo limitations involving the use of TCSAdvocacy
Bosdriesz et al.32 2015, the NetherlandsTo assess whether developments in tobacco control policy in the Netherlands were associated with smoking cessation and smoking intensity.Observational, multilevel, and longitudinalIndependent variable, not original data, only by components scoreProgress in tobacco control policy in the Netherlands was significantly associated with an increase in the quit ratios (2001–2011) but were not significantly associated with smoking intensity among smokers. The strength of the associations was similar for low- and high-education groupsNo limitations reported about the TCS as a variableAdvocacy and research
Bosdriesz et al.30 2015, the NetherlandsTo assess variations in the progress of tobacco control policy development in Europe and to identify whether the variations can be decomposed into specific patterns or componentsObservational, ecological, and longitudinalDependent variable, not original, only by components scoreProgress in tobacco control policy development in Europe was not uniform. Consistent progress was observed in several areas but was lacking in tobacco prices and smoking cessation supporta. TCS score sometimes fails to express the degree to which policies are enforced; b. some policy areas could not be quantified readily; c. not able to include each separate measure of the TCS in its own rightResearch
Klumbiene et al.29 2015, LithuaniaTo evaluate the association between tobacco control policies and trends in smoking cessation in Lithuania in 1994–2010Observational, ecological, and longitudinalIndependent variable, not original data, total score and by componentsGreat progress in the development of tobacco control policy has been achieved in Lithuania. This progress was associated with an increase in smoking cessation. This association was stronger among younger than older peopleNo limitations reported about the TCS as a variableResearch
Kuipers et al.28 2015, the NetherlandsTo examine the association between tobacco control policies and adolescent smoking, and to investigate the differences in this association between adolescents of high and low socio-economic status (SES)Observational, multilevel, and cross-sectionalIndependent variable, not original data, only total scoreAdolescent smoking prevalence rates were higher among low-SES respondents than their high-SES peers. Stronger national-level tobacco control policies were associated with lower odds of daily smokingThe TCS used in the current study contains five domains of tobacco control, not all of which may be as likely to affect adolescent smokingAdvocacy and research
Bosdriesz et al.6 2015, the NetherlandsTo provide insight into the role of political factors in the development of tobacco control policy over timeObservational, ecological, and longitudinalDependent variable, not original, total score and by componentsAn association was found between left-wing government and TCS over the period 1996–2003, but not over the whole studied period (1996–2010). The association between government effectiveness and TCS was significant and negative over the whole period, but positive between 2001 and 2005The TCS contains little information on their enforcement in practiceAdvocacy
Movsisyan et al.33 2014, ArmeniaTo measure the 5-year progress in the implementation of FCTC in ArmeniaObservational, ecological, and cross-sectionalDependent variable, not original, total score and by componentsThe estimated TCS score for Armenia for smoke-free public places, advertising ban, health warnings, and treatment are below the European average (2005–2007). However, the score estimate for price and public spending are above averagea. Potential measurement error; b. inadequate accuracy and comparability of data; c. the estimates could have been affected by exchange rate fluctuationsAdvocacy and research
Heydari et al.34 2012, IranTo obtain an overview of tobacco control strategies in the Eastern Mediterranean regionObservational, ecological, and cross-sectionalDependent variable, not original, total score and by componentsAfghanistan scored highest for tobacco pricing. Oman scored higher than others for regulations and enforcement of bans on smoking in public places. The Islamic Republic of Iran had the top score on budgeting for tobacco control activities, in prohibition and enforcement of tobacco advertising, and placement of health warnings on cigarette packets. Syrian Arab Republic, Tunisia, and Kuwait had the best provision of smoking cessation services, whereas Pakistan, Saudi Arabia, Somalia, and Yemen scored zeroAs the data were extracted from sources such as MPOWER measures and the Tobacco Atlas, they may not cover all important variables and the results may not be conclusiveResearch

Twelve out of 21 articles (60%) using individual TCS scores included all six policy components in the analysis. The most frequently used policy components were the individual score on bans in public places (n=20; 95.2%) and advertising bans (n=16; 76.2%). In contrast, the least used were data on public spending on information campaigns (n=12; 57.1%). The publications included data from between 1 and 31 countries; only one publication used scores from all of the countries included in the TCS report, including >30 EU and non-EU countries7; however, 46.9% of publications included scores from all EU Member states except Croatia because it was first included in 2013.

Half (n=16) of the publications were directed towards policymakers with the aim of urging governments to implement more stringent tobacco control policies, 6 publications aimed to foster further research on this topic (18.8%), and the conclusions of 10 papers (31.3%) addressed both aims.

Articles using TCS scores from primary reports

Almost all of the studies that used the TCS reports as a primary source (n=22) (Table 2), were observational in nature (n=21; 95.5%) and 19 were cross-sectional (86.4%). According to the type of unit of analysis, half were ecological studies and half multilevel. These studies aimed to address the relationship between tobacco control policies and several outcomes, such as the prevalence of preterm births and low birthweight7, of hard-core and light smokers8, and of smoking in adolescents9,10; smoking prevalence and quit ratios4,11; consumption of rolling tobacco, e-cigarettes, and readiness to quit in adults12; and risk of lung cancer13. Other indicators were smoking in private venues1,14, self-reported exposure to secondhand smoke (SHS)15-17, and attitudes towards smoking and tobacco product restrictions18-21.

Other publications were focused on exploring the association between the price of tobacco products and infant mortality22, the effects of sales restriction laws on adolescents23, and the factors associated with exposure to tobacco and e-cigarette advertising24. One study assessed the association between smoking prevalence and public sector corruption and other national characteristics25. The main characteristics and results of each article are given in Table 2.

Articles using TCS methodology to compute new scores

As shown in Table 3, ten studies calculated new scores to measure tobacco control policies at a country level in a particular year using the TCS rationale and methodology instead of the original TCS. Most of these studies used data from European countries with a longitudinal design aimed at assessing the association between tobacco control policies and smoking6,26,27 and socio-economic inequalities outcomes in adolescents28 or adults over time29-31, or to examine political factors that drive tobacco control policy development32. According to the type of data, these publications were half ecological and half multilevel studies.

Two publications computed scores for nonEuropean countries to monitor their tobacco control policy implementation by using the same rationale and methodology. These publications were aimed at measuring the progress after implementation of the WHO-FCTC in Armenia33 and providing an overview and comparing the tobacco control progress in Eastern Mediterranean countries34. The main characteristics and results of each study are shown in Table 3.

Main limitations of the TCS mentioned by the studies

Only 11 (34.4%) of all publications included comments on the limitations of using the TCS score as a tool to assess tobacco control policies. The main limitations reported by the studies were that they failed to express the degree to which legislative policies are enforced6,32,35, except for the smoke-free policies4. Another limitation is that the countries’ rankings have only slightly changed over the years (i.e. the UK has remained in the top position from 2007 to 2016)4. This low variance across countries may reduce the robustness of the results of the studies10. Moreover, some studies reported that the information described by the TCS score(s) does not incorporate the most recent national legislation on tobacco control due to its cross-sectional design13, potentially underestimating the impact of such policies when using the TCS11.

Finally, among the studies not using data from the original TCS reports, the main limitations were that some policy areas could not be quantified accurately and that some of the policy components assessed by the TCS could not be included34 because of potential error in the measurement of their estimates, and inadequate accuracy and comparability of the data33.

DISCUSSION

Our results reveal that the TCS has been used mostly in observational, cross-sectional studies with either ecological (country as the unit of analysis) or multilevel data (individual data from surveys as the first-level unit and TCS score by country as secondlevel aggregated information). The TCS score has been mainly used as an independent variable to explain the potential variation in outcomes (i.e. tobacco product use, exposure to SHS, attitudes towards legislation, etc.), and mostly employed in European countries, as these countries were the target of the TCS when it was created.

Interpretation of the results

This is the first attempt to assess all of the available publications that have used the TCS as a means to measure tobacco control policy implementation since it was developed in 2006. In addition, this is the first study to map out the characteristics of the use of the TCS in scientific research, to understand how this tool has been applied despite its original design as a means to advocate for comprehensive tobacco control policies. Therefore, our findings suggest that the TCS has commonly been used as an indicator of the state of tobacco control policies in Europe.

Almost all of the studies assessed tobacco control policies through the total TCS score, and most have used the policy components scores from the primary published reports. The policy components most commonly studied were public smoking bans and tobacco product advertising bans, possibly because they are two of the measures that have been most frequently regulated in Europe since the WHO-FCTC came into force in 200536.

Another important issue to address is the crosssectional and temporal comparability of the TCS because most of the studies make comparisons across countries and/or over time. Notably, Joossens and Raw5 designed the scale to compare tobacco control policies across countries at a particular time point. Thus, the reference values for scores are sustained across each report. However, these scores are not comparable across years because these standards have changed over time (i.e. the weighted average price for cigarettes was €8.5 in 2013 and €10 in 2016, considering the EU average Purchasing Power Standard)37,38; on the other hand, the scale methodology and scoring system changed between 2007 and 201039. Consequently, longitudinal studies to ensure temporal comparability between and within countries require adjusting scores to the highest standards by an escalation process and re-calculating scores from the 2005 and 2007 reports using the newest scoring system and methodology.

Importantly, most of the studies with a longitudinal design conducted in Europe have adapted the scale rationale and methods to estimate the level of implementation of tobacco control policies to ensure the temporal comparability and include data about years for which the TCS had not been published26,27,29,31,32. Few non-European countries have adapted the scale as a proxy to monitor the status of tobacco control policies in non-European countries33,34. Unfortunately, most of these studies did not clearly explain how they adapted the TCS to estimate new score(s) for each policy component. Therefore, new studies should provide a full description of their adaptation process and the potential limitations and strengths not only to ensure its replication, but also to further develop strategies to adapt the TCS to other contexts overseas.

These results highlight that the TCS, regardless of its limitations, has been applied as an objective indicator to measure the strength of the implementation of tobacco control policies at the country level. Other studies have used a total score obtained from summing the scores (from 1 to 5) assigned to each MPOWER policy dimension in the WHO’s Reports on the Global Tobacco Epidemic40,41. MPOWER’s composite score has some clear advantages over the TCS total score because it is available for all countries, not only European countries, and is comparable over time. However, this proxy also has some disadvantages for research purposes. First, it assigns the same weight to each of the six individual MPOWER scores without taking into consideration that some MPOWER measures have been proven to be more effective than other measures (i.e. taxation). Second, MPOWER’s composite score has a narrower score range than the TCS score (6–29 vs 0–100, respectively), which limits variation across countries and may make it difficult to address variability between countries. Finally, unlike the MPOWER composite score, the TCS score is not affected by the government’s political agenda, as the TCS is built on information from objective databases (i.e. Eurostat) and the Civil Society.

More than 65% of the reviewed publications did not report any limitation of the TCS as a proxy for measuring tobacco control policies. Nonetheless, Joossens and Raw5 already reported some of its major limitations, including difficulties in assessing enforcement versus implementation and its critical dependence on tobacco control experts’ judgement when scoring5. Therefore, such underreporting of limitations makes it difficult to fully describe the limitations that researchers encounter, which is indispensable to moving forward in the field. Among the articles reporting limitations, most of them highlighted the fact that the TCS score(s) did not measure the enforcement of policies except smokefree policies. In this sense, no previous studies have examined the disparity between the implementation and enforcement of tobacco control policies; however, the TCS being predictive of so many outcomes suggests that the implementation of these policies is a good proxy of enforcement. In addition, some studies have questioned the ability of the TCS to incorporate new policies, such as smoke-free outdoor policies, indicating that the authors of the TCS should discuss how to incorporate these new tobacco control policies and which weight they should have in the scale.

Moreover, our study shows that the TCS has been commonly used in Europe over the last decade, but three research groups from Spain, the Netherlands, and the UK account for more than half of the publications using the TCS for research purposes. This suggests that these groups have led and consolidated the use of this monitoring tool in the tobacco control research field. This is supported by the fact that the publications conducted by these three research groups have received a higher number of citations, including a paper13 with 54 citations and another paper23 with 103 citations (in Web of Science) up to December 2019.

More than half of all publications directed their conclusions towards advocacy for improving tobacco control policies. Therefore, most authors find the TCS useful for linking data to policy action, even though the TCS has been commonly used for research purposes. Therefore, the TCS has not lost its intended original purpose for advocacy, as it was developed to detect areas of improvement within each country and to establish comparisons among countries through a ranking, in order to motivate governments to strengthen their weakest polices5.

Our results indicate that, despite its potential limitations and lack of a formal validity assessment, the TCS is a good proxy of the strength of tobacco control policies implementation, or at least the best approximation developed so far. However, the TCS has been used at face-value. No attempts have been made to formally validate the scale. Construct validity of the TCS is a complex issue given the composite structure of the TCS itself, though some dimensions are based on objective data (i.e. price and SHS exposure) from population-based surveys and reports of the European Commission; others are based on the answers of one or two informants to an ad hoc questionnaire (i.e. cessation budget at national level)5.

Limitations and strengths

Publication bias is a potential source of error when the units of the investigation are published papers42. We searched the available literature in PubMed, the main biomedical database, as well as Web of Science and Google Scholar, and checked all references to identify other articles not published in academic journals. However, the possibility that unpublished manuscripts or other documents addressing the topic of interest may have been missed cannot be ruled out, but it was an a priori decision made by the experienced research team that was composed of tobacco control and policy experts, including the author of the TCS. Under these circumstances, selection (publication) bias seems unlikely to have affected the study.

Other potential limitations of our study are linked to the fact that a high number of the publications analyzed here did not include any comment about the limitations and strengths of using TCS scores as a variable to monitor tobacco control policy implementation in research. This missed reporting has hindered the identification of the main limitations and strengths of this tool for different types of study designs, outcomes, or statistical analysis; therefore, our study may have some missing information.

However, our study is the first to assess all published articles using the TCS as an indicator of tobacco control policy implementation and to characterize its use in tobacco control research, giving a full and comprehensive overview of how and for which purposes the TCS has been employed in previous studies. This study also presents information on how to best use the TCS as described by the authors in the Limitations sections of the publications. Thus, our study presents the lessons learned from previous research, creating an opportunity for researchers to plan to use the TCS to improve the quality of future studies.

CONCLUSIONS

This study shows that the TCS has been commonly used in observational, mostly ecological, studies to assess variations in a concrete outcome according to the policies instituted in Europe as a proxy of tobacco control implementation. In addition, the TCS has been employed to detect changes in individual and population outcomes (i.e. smoking prevalence or cessation) and establish conclusions about how policies have an effect in specific populations.

Our recommendations to researchers and policymakers planning to use the TCS in their future research are as follows. First, the TCS scoring methodology needs to be fully understood, as comparability is not ensured among countries across years. Second, researchers should consider a certain time gap between measuring the TCS score and the outcomes, as the TCS may not include the most recently adopted policies and policies need time to have an effect. Third, knowing the limitations of the TCS in measuring implementation (vs enforcement) of tobacco control policies is important. Fourth, researchers need to take into account the low variance of some tobacco control policies across countries, which may also reduce the robustness of the estimates.

A logical next step for future applications of the TCS in research would be to study the impact of tobacco control policy enforcement in terms of several indicators, such as prevalence, SHS, and tobacco sales, and to assess the impact of these policies at the population level. To achieve this goal, more extensive cross-country population-based surveys are needed to include new enforcement measures in future editions of the TCS (i.e. about compliance with smoke-free bans in public places differently than workplaces and hospitality venues, or about advertising, promotion, and sponsorship bans).

Finally, to gain a broader perspective of tobacco control as a public health need and build a stronger tool for tobacco research, we suggest adapting and extending the TCS to other countries of the WHO European Region, and to the reality of other regions of the globe, such as Latin America or Asia, incorporating local and cultural characteristics of these regions while preserving the comparability among countries worldwide.

CONFLICTS OF INTEREST

The authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none was reported.

FUNDING

AF, MF, CM, and EF were supported by the Ministry of Research and Universities from the Government of Catalonia [2017SGR319]. EF was also supported by the Instituto de Salud Carlos III, Government of Spain, co-funded by the European Regional Development Fund (FEDER) [INT16/00211 and INT17/00103]. CM was also supported by the Instituto de Salud Carlos III, Government of Spain, co-funded by the European Regional Development Fund (FEDER) [INT17/00116] and Ministry of Health from the Government of Catalonia [PERIS No. 9015586920/2017]. We thank CERCA Programme/Generalitat de Catalunya for institutional support.

AUTHORS’ CONTRIBUTIONS

Study design: AF, AB, CM, and EF. Collected data and prepared database for analysis: AF and AB. Contributed to strategy of analysis: AF, AB, CM, and EF. Analyzed data: AF. Interpreted data results: AF, AB, CM, LJ, MF, AP, and EF. Drafted manuscript: AF. Critically revised manuscript: All authors. Approved final manuscript version: All authors. Guarantors: EF and CM.

PROVENANCE AND PEER REVIEW

Not commissioned; externally peer reviewed.