The Tobacco Control Scale as a research tool to measure country-level tobacco control policy implementation

INTRODUCTION The Tobacco Control Scale (TCS) was designed for advocacy purposes but has also been used as a research tool. In the present study, we characterized TCS use, its limitations and strengths, and critically assessed its use as a research instrument. METHODS We conducted an extensive search of the biomedical databases PubMed and Web of Science for the keyword ‘tobacco control scale’ in all fields. The search was limited to studies published in the period March 2006 to December 2019. Out of 69 hits, 32 studies met the inclusion criteria. Two reviewers independently extracted information from each publication regarding their general characteristics, publication and research aspects, and the characteristics of the use of the TCS. RESULTS We found that researchers have used the TCS as a tool to monitor tobacco control policies mainly in cross-sectional observational studies with ecological and multilevel designs directed to advocacy and the promotion of further research. Different outcomes, such as smoking prevalence and quit ratios, have been associated with tobacco control policy scores. The main reported limitations of the TCS were a low variance across countries and a failure to express enforcement and to incorporate the most recent legislation. CONCLUSIONS The TCS has been commonly used to assess differences in outcomes according to tobacco control policies. However, there are still areas for improvement in its use in research regarding the lack of comparability of TCS scores across time. The lessons that have been learned should be used to adapt and expand the TCS overseas.

have been made globally to tackle the tobacco epidemic 3 , stimulated by the enforcement of the Framework Convention on Tobacco Control. In the European Union (EU), the Tobacco Products Directive has driven the application of stringent tobacco control policies to reduce tobacco use and its negative consequences on health. However, the implementation and enforcement of tobacco control policies still vary greatly across Europe 4 .
Among the initiatives to monitor the implementation of tobacco control policies in Europe, Joossens and Raw 5 developed the Tobacco Control Scale (TCS) in 2006. The TCS score is determined by a questionnaire based on six cost-effective policy interventions that should be prioritized according to the World Bank. These measures include taxation, smoke-free policies, public spending in information campaigns, advertising bans, health warnings, and treatment. The score assigned to each of these components is weighted by its reported evidence-based effectiveness. Therefore, the score attributed to each country increases with the strength of tobacco control policies up to a maximum of 100 points, indicating full implementation 5 .
At its inception, the aim of the TCS was to monitor the progress in tobacco control in Europe at a national level by comparing the performance of countries by their ranking 5 and to inform the agenda by highlighting the policy components for which progress is lacking, as well as the countries or regions most affected by such delays 6 . Since 2006, the TCS has been updated every three years (available at www. tobaccocontrolscale.org).
Evaluating the impact of tobacco control policies among the population has become an important research area; thus, the TCS has been used as a research tool to measure the implementation of tobacco control policies, though it was not designed for such purposes. However, little is known about the use of the TCS by the tobacco control research community and its advantages and disadvantages as a research tool. Therefore, our aim was to characterize the use of the TCS by researchers and its main limitations and strengths as a research tool in order to critically assess its use as a research instrument.

Data sources
We performed an extensive literature search in the online databases PubMed and Web of Science to identify publications that have used the TCS score(s) as an independent or dependent variable from 27 March 2006, when the first TCS was published, until 1 December 2019. The search was conducted using 'tobacco control scale' as the keyword in all fields without any other restrictions to ensure a very sensitive search. The Ethics and Clinical Research Committee of the Hospital Universitari de Bellvitge approved this study (PR247/18).

Study selection
We identified 69 publications (32 duplicated in both databases). After removing duplicates, two researchers (AF and AB) screened the titles and abstracts, obtaining 32 studies. The inclusion criteria were quantitative research and inclusion of the TCS score(s) (as dependent or independent variable) in the analysis. We found 27 eligible publications ( Figure 1). We completed our search by manually reviewing the reference lists of the selected papers and by conducting the same search in Google Scholar (www.scholar.google.com; with search terms in English). These additional searches provided five new publications that met the inclusion criteria and the full-texts were reviewed. Therefore, we finally included 32 publications that used the TCS score(s) as a dependent or independent variable.

Data extraction
A detailed protocol and Microsoft Access® database were designed to extract and register the information from each publication. The evaluation protocol was developed by three researchers who are experts in tobacco control (AF, CM, and EF). The protocol describing the main objectives, information sources, search strategy and eligibility criteria, and the data collection was reviewed and approved by all researchers. All variables for which data were described in the publications' Methods sections were listed.
Two reviewers independently extracted the data according to the protocol (AF and AB). If any discrepancies emerged, the reviewers discussed the papers until agreement was reached and, when no consensus was met, divergences were solved by discussing them with a third reviewer (CM). The evaluation process was conducted in January 2020.
The extracted information included general characteristics, publication characteristics, research characteristics, and characteristics of the use of the TCS (Table 1).
Most of the publications (n=31) were observational studies; 16 used ecological data (50%) with the country as the unit of analysis and 16 used multilevel data (50%) with individual data from surveys as the first-level unit and TCS score by country as second-level aggregated information. Regarding the study design, 23 of the publications were crosssectional studies (78.6%). Most of the articles (n=24; 75%) ( Table 2) included the TCS score(s) as an independent variable from primary reports, whereas 10 publications (31.3%) ( Table 3) used the scores from secondary sources that calculated a new score based on the TCS methodology. Overall, 87.5% (n=28) of the publications used the overall TCS score by country and 65.6% (n=21) used the individual policy component scores.
Twelve out of 21 articles (60%) using individual TCS scores included all six policy components in the analysis. The most frequently used policy components were the individual score on bans in public places (n=20; 95.2%) and advertising bans (n=16; 76.2%). In contrast, the least used were data on public spending on information campaigns (n=12; 57.1%). The publications included data from between 1 and 31 countries; only one publication used scores from all of the countries included in the TCS report, including >30 EU and non-EU countries 7 ; however, 46.9% of publications included scores from all EU Member states except Croatia because it was first included in 2013.  Half (n=16) of the publications were directed towards policymakers with the aim of urging governments to implement more stringent tobacco control policies, 6 publications aimed to foster further research on this topic (18.8%), and the conclusions of 10 papers (31.3%) addressed both aims.

Articles using TCS scores from primary reports
Almost all of the studies that used the TCS reports as a primary source (n=22) ( Table 2), were observational in nature (n=21; 95.5%) and 19 were cross-sectional (86.4%). According to the type of unit of analysis, half were ecological studies and half multilevel. These studies aimed to address the relationship between tobacco control policies and several outcomes, such as the prevalence of preterm births and low birthweight 7 , of hard-core and light smokers 8 , and of smoking in adolescents 9,10 ; smoking prevalence and quit ratios 4,11 ; consumption of rolling tobacco, e-cigarettes, and readiness to quit in adults 12 ; and risk of lung cancer 13 . Other indicators were smoking in private venues 1,14 , self-reported exposure to secondhand smoke (SHS) [15][16][17] , and attitudes towards smoking and tobacco product restrictions [18][19][20][21] .
Other publications were focused on exploring the association between the price of tobacco products          22 , the effects of sales restriction laws on adolescents 23 , and the factors associated with exposure to tobacco and e-cigarette advertising 24 .
One study assessed the association between smoking prevalence and public sector corruption and other national characteristics 25 . The main characteristics and results of each article are given in Table 2.
Articles using TCS methodology to compute new scores As shown in Table 3, ten studies calculated new scores to measure tobacco control policies at a country level in a particular year using the TCS rationale and methodology instead of the original TCS. Most of these studies used data from European countries with a longitudinal design aimed at assessing the association between tobacco control policies and smoking 6,26,27 and socio-economic inequalities outcomes in adolescents 28 or adults over time [29][30][31] , or to examine political factors that drive tobacco control policy development 32 .
According to the type of data, these publications were half ecological and half multilevel studies. Two publications computed scores for non-European countries to monitor their tobacco control policy implementation by using the same rationale and methodology. These publications were aimed at measuring the progress after implementation of the WHO-FCTC in Armenia 33 and providing an overview and comparing the tobacco control progress in Eastern Mediterranean countries 34 . The main characteristics and results of each study are shown in Table 3.

Main limitations of the TCS mentioned by the studies
Only 11 (34.4%) of all publications included comments on the limitations of using the TCS score as a tool to assess tobacco control policies. The main limitations reported by the studies were that they failed to express the degree to which legislative policies are enforced 6,32,35 , except for the smoke-free policies 4 . Another limitation is that the countries' rankings have only slightly changed over the years (i.e. the UK has remained in the top position from 2007 to 2016) 4 . This low variance across countries may reduce the robustness of the results of the studies 10 . Moreover, some studies reported that the information described by the TCS score(s) does not incorporate the most recent national legislation on tobacco control due to its cross-sectional design 13 , potentially underestimating the impact of such policies when using the TCS 11 .
Finally, among the studies not using data from the original TCS reports, the main limitations were that some policy areas could not be quantified accurately and that some of the policy components assessed by the TCS could not be included 34 because of potential error in the measurement of their estimates, and inadequate accuracy and comparability of the data 33 .

DISCUSSION
Our results reveal that the TCS has been used mostly in observational, cross-sectional studies with either ecological (country as the unit of analysis) or multilevel data (individual data from surveys as the first-level unit and TCS score by country as secondlevel aggregated information). The TCS score has been mainly used as an independent variable to explain the potential variation in outcomes (i.e. tobacco product use, exposure to SHS, attitudes towards legislation, etc.), and mostly employed in European countries, as these countries were the target of the TCS when it was created.

Interpretation of the results
This is the first attempt to assess all of the available publications that have used the TCS as a means to measure tobacco control policy implementation since it was developed in 2006. In addition, this is the first study to map out the characteristics of the use of the TCS in scientific research, to understand how this tool has been applied despite its original design as a means to advocate for comprehensive tobacco control policies. Therefore, our findings suggest that the TCS has commonly been used as an indicator of the state of tobacco control policies in Europe.
Almost all of the studies assessed tobacco control policies through the total TCS score, and most have used the policy components scores from the primary published reports. The policy components most commonly studied were public smoking bans and tobacco product advertising bans, possibly because they are two of the measures that have been most frequently regulated in Europe since the WHO-FCTC came into force in 2005 36 .
Another important issue to address is the crosssectional and temporal comparability of the TCS because most of the studies make comparisons across countries and/or over time. Notably, Joossens and Raw 5 designed the scale to compare tobacco control policies across countries at a particular time point. Thus, the reference values for scores are sustained across each report. However, these scores are not comparable across years because these standards have changed over time (i.e. the weighted average price for cigarettes was €8.5 in 2013 and €10 in 2016, considering the EU average Purchasing Power Standard) 37,38 ; on the other hand, the scale methodology and scoring system changed between 2007 and 2010 39 . Consequently, longitudinal studies to ensure temporal comparability between and within countries require adjusting scores to the highest standards by an escalation process and re-calculating scores from the 2005 and 2007 reports using the newest scoring system and methodology.
Importantly, most of the studies with a longitudinal design conducted in Europe have adapted the scale rationale and methods to estimate the level of implementation of tobacco control policies to ensure the temporal comparability and include data about years for which the TCS had not been published 26,27,29,31,32 . Few non-European countries have adapted the scale as a proxy to monitor the status of tobacco control policies in non-European countries 33,34 . Unfortunately, most of these studies did not clearly explain how they adapted the TCS to estimate new score(s) for each policy component. Therefore, new studies should provide a full description of their adaptation process and the potential limitations and strengths not only to ensure its replication, but also to further develop strategies to adapt the TCS to other contexts overseas.
These results highlight that the TCS, regardless of its limitations, has been applied as an objective indicator to measure the strength of the implementation of tobacco control policies at the country level. Other studies have used a total score obtained from summing the scores (from 1 to 5) assigned to each MPOWER policy dimension in the WHO's Reports on the Global Tobacco Epidemic 40,41 . MPOWER's composite score has some clear advantages over the TCS total score because it is available for all countries, not only European countries, and is comparable over time. However, this proxy also has some disadvantages for research purposes. First, it assigns the same weight to each of the six individual MPOWER scores without taking into consideration that some MPOWER measures have been proven to be more effective than other measures (i.e. taxation). Second, MPOWER's composite score has a narrower score range than the TCS score (6-29 vs 0-100, respectively), which limits variation across countries and may make it difficult to address variability between countries. Finally, unlike the MPOWER composite score, the TCS score is not affected by the government's political agenda, as the TCS is built on information from objective databases (i.e. Eurostat) and the Civil Society.
More than 65% of the reviewed publications did not report any limitation of the TCS as a proxy for measuring tobacco control policies. Nonetheless, Joossens and Raw 5 already reported some of its major limitations, including difficulties in assessing enforcement versus implementation and its critical dependence on tobacco control experts' judgement when scoring 5 . Therefore, such underreporting of limitations makes it difficult to fully describe the limitations that researchers encounter, which is indispensable to moving forward in the field. Among the articles reporting limitations, most of them highlighted the fact that the TCS score(s) did not measure the enforcement of policies except smokefree policies. In this sense, no previous studies have examined the disparity between the implementation and enforcement of tobacco control policies; however, the TCS being predictive of so many outcomes suggests that the implementation of these policies is a good proxy of enforcement. In addition, some studies have questioned the ability of the TCS to incorporate new policies, such as smoke-free outdoor policies, indicating that the authors of the TCS should discuss how to incorporate these new tobacco control policies and which weight they should have in the scale.
Moreover, our study shows that the TCS has been commonly used in Europe over the last decade, but three research groups from Spain, the Netherlands, and the UK account for more than half of the publications using the TCS for research purposes. This suggests that these groups have led and consolidated the use of this monitoring tool in the tobacco control research field. This is supported by the fact that the publications conducted by these three research groups have received a higher number of citations, including a paper 13 with 54 citations and another paper 23 with 103 citations (in Web of Science) up to December 2019.
More than half of all publications directed their conclusions towards advocacy for improving tobacco control policies. Therefore, most authors find the TCS useful for linking data to policy action, even though the TCS has been commonly used for research purposes. Therefore, the TCS has not lost its intended original purpose for advocacy, as it was developed to detect areas of improvement within each country and to establish comparisons among countries through a ranking, in order to motivate governments to strengthen their weakest polices 5 .
Our results indicate that, despite its potential limitations and lack of a formal validity assessment, the TCS is a good proxy of the strength of tobacco control policies implementation, or at least the best approximation developed so far. However, the TCS has been used at face-value. No attempts have been made to formally validate the scale. Construct validity of the TCS is a complex issue given the composite structure of the TCS itself, though some dimensions are based on objective data (i.e. price and SHS exposure) from population-based surveys and reports of the European Commission; others are based on the answers of one or two informants to an ad hoc questionnaire (i.e. cessation budget at national level) 5 .

Limitations and strengths
Publication bias is a potential source of error when the units of the investigation are published papers 42 . We searched the available literature in PubMed, the main biomedical database, as well as Web of Science and Google Scholar, and checked all references to identify other articles not published in academic journals. However, the possibility that unpublished manuscripts or other documents addressing the topic of interest may have been missed cannot be ruled out, but it was an a priori decision made by the experienced research team that was composed of tobacco control and policy experts, including the author of the TCS. Under these circumstances, selection (publication) bias seems unlikely to have affected the study.
Other potential limitations of our study are linked to the fact that a high number of the publications analyzed here did not include any comment about the limitations and strengths of using TCS scores as a variable to monitor tobacco control policy implementation in research. This missed reporting has hindered the identification of the main limitations and strengths of this tool for different types of study designs, outcomes, or statistical analysis; therefore, our study may have some missing information.
However, our study is the first to assess all published articles using the TCS as an indicator of tobacco control policy implementation and to characterize its use in tobacco control research, giving a full and comprehensive overview of how and for which purposes the TCS has been employed in previous studies. This study also presents information on how to best use the TCS as described by the authors in the Limitations sections of the publications. Thus, our study presents the lessons learned from previous research, creating an opportunity for researchers to plan to use the TCS to improve the quality of future studies.

CONCLUSIONS
This study shows that the TCS has been commonly used in observational, mostly ecological, studies to assess variations in a concrete outcome according to the policies instituted in Europe as a proxy of tobacco control implementation. In addition, the TCS has been employed to detect changes in individual and population outcomes (i.e. smoking prevalence or cessation) and establish conclusions about how policies have an effect in specific populations.
Our recommendations to researchers and policymakers planning to use the TCS in their future research are as follows. First, the TCS scoring methodology needs to be fully understood, as comparability is not ensured among countries across years. Second, researchers should consider a certain time gap between measuring the TCS score and the outcomes, as the TCS may not include the most recently adopted policies and policies need time to have an effect. Third, knowing the limitations of the TCS in measuring implementation (vs enforcement) of tobacco control policies is important. Fourth, researchers need to take into account the low variance of some tobacco control policies across countries, which may also reduce the robustness of the estimates.
A logical next step for future applications of the TCS in research would be to study the impact of tobacco control policy enforcement in terms of several indicators, such as prevalence, SHS, and tobacco sales, and to assess the impact of these policies at the population level. To achieve this goal, more extensive cross-country population-based surveys are needed to include new enforcement measures in future editions of the TCS (i.e. about compliance with smoke-free bans in public places differently than workplaces and hospitality venues, or about advertising, promotion, and sponsorship bans).
Finally, to gain a broader perspective of tobacco control as a public health need and build a stronger tool for tobacco research, we suggest adapting and extending the TCS to other countries of the WHO European Region, and to the reality of other regions of the globe, such as Latin America or Asia, incorporating local and cultural characteristics of these regions while preserving the comparability among countries worldwide.