Study design

This study employed a concurrent mixed-methods design, combining cross-sectional quantitative surveys with qualitative focus group interviews¹³. This methodological approach was chosen to achieve triangulation, with quantitative message effectiveness rankings complementing and corroborating specific feedback from young people in interviews to identify effective message characteristics and content elements¹⁴.

Participants

This study employed purposive sampling and was conducted in March 2021 in primary schools, middle schools, high schools, and universities in Beijing and Kunming. All participating institutions were public schools and universities. These two cities were selected to maximize sample diversity and enable comparison. Beijing, located in northern China, has performed relatively well in tobacco control work and has implemented smoke-free regulations compliant with the WHO Framework Convention on Tobacco Control, with relatively high public tobacco control awareness¹⁵. Kunming, located in southern China and the capital of Yunnan Province, represents a region where the local economy is highly dependent on tobacco, as Yunnan is a major tobacco production and consumption province with relatively low public tobacco control awareness¹⁶.

The target recruitment was approximately 40 participants per educational level (primary school, middle school, high school, and university), with equal representation across cities and genders within each level. A total of 147 participants were recruited from 172 eligible students, divided into 16 groups by city, school level, and gender, with 8–10 participants per group. The final sample included 35 primary school, 41 middle school, 49 high school, and 22 university participants. Inclusion criteria were: 1) currently enrolled in primary schools, middle schools, high schools, or universities in Beijing or Kunming; 2) aged 11–23 years; 3) having adequate comprehension abilities; and 4) for participants under 18 years of age, parent/guardian agreeing to participate and signing informed consent, for participants aged 18 years and older, written informed consent was obtained directly from the participants themselves. For participants under 18 years of age, written informed consent was obtained from a parent or legal guardian, and assent was obtained from the participant. The participation rate was 85.5% (147 of 172 eligible students). The study protocol received ethical approval from the Institutional Review Board of Peking University (IRB00001052-20056).

Data collection procedures

The mixed-methods procedure followed a systematic sequential approach within each data collection session. Participants first individually reviewed all ten tobacco control messages presented in randomized order and completed quantitative evaluations using structured questionnaires. Following completion of the quantitative assessment phase, participants were organized into focus groups of 6–8 individuals stratified by city, school level, and gender. Trained facilitators conducted semi-structured interviews using a standardized guide to explore the underlying reasons for message effectiveness ratings, ensuring all participants had opportunities to contribute their perspectives on what made specific messages compelling or ineffective for tobacco prevention. This sequential approach ensured that qualitative insights could build upon and contextualize quantitative findings while avoiding potential bias from group discussions influencing individual message ratings¹³.

Test materials

Ten text-based tobacco control messages were selected for evaluation based on a comprehensive review of existing Chinese tobacco control campaigns, international best practices, and consultation with public health experts. The messages represented three distinct categories.

Health consequence messages (M1–M7)

Seven messages provided factual information about specific health risks of smoking and secondhand smoke exposure², including: nicotine addiction (M1), severe lung diseases and COPD (M2), periodontitis (M3), multiple cancers (M4), otitis media (M5), asthma (M6), and coronary heart disease and stroke (M7). These messages varied in the familiarity, severity, and specificity of the health conditions described.

Directive messages (M8, M10)

Two messages used imperative or instructive language: ‘Refuse the first cigarette’ (M8) and ‘Create a smoke-free campus, be a non-smoking new generation – please don’t tempt me to smoke’ (M10). These messages are commonly used in Chinese tobacco control campaigns and represent traditional directive approaches to prevention⁹.

Positive framing message (M9)

One message attempted to frame non-smoking in positive terms: ‘No smoking, I’m healthy, I’m fashionable’ (M9). This message represents attempts to associate non-smoking with desirable identity characteristics.

The complete text of all ten messages in both Chinese and English is provided in Supplementary file Table 1.

Questionnaire content

Data collection occurred in classroom settings. The questionnaire included: 1) Demographic variables: gender (male, female), city (Beijing, Kunming), school type (primary school, middle school, high school, university), and age; 2) Behavioral assessment: current smoking status (yes, no), current e-cigarette use (yes, no), living with smoker(s) (yes, no), indoor smoking permitted at home (yes, no), and having friends who smoke (yes, no); and 3) Message effectiveness evaluation: participants selected the one message that would be most likely to prevent them from starting to smoke and the one message that would be least likely to do so. This ensured independent ratings without peer influence.

Following completion of the quantitative assessment phase, participants were organized into focus groups. Focus group discussions lasting 60–90 minutes explored participants’ reasoning for selecting certain messages as most or least effective, what elements made messages convincing or unconvincing, and their overall impressions of different message types. The facilitators explained ‘effective’ as a message that may discourage the participant from starting to smoke (indicating positive deterrent potential), and ‘ineffective’ as a message that would be unlikely to discourage smoking initiation (indicating limited deterrent potential)¹¹. Discussions were audio-recorded and professionally transcribed verbatim in Chinese.

Data analysis

Quantitative analysis

Descriptive statistics characterized the sample. The effectiveness index (EI) was calculated for each message as: EI = (% rating message as most effective) - (% rating message as least effective)¹⁷. This descriptive metric provides an intuitive measure of overall perceived effectiveness, with positive scores indicating net effectiveness and negative scores indicating net ineffectiveness. Chi-squared tests of independence examined associations between demographic variables (city, gender, school type) and message selection, with p<0.05 considered statistically significant. To assess whether demographic factors independently predicted message selection, binary logistic regression analyses were performed. To determine the most effective message, we examined whether demographic variables predicted selection of the top-ranked message (M4: cancer message) over all other messages. For the least effective message, we examined predictors of selecting M9 (fashionable message) over all other messages. Both crude (unadjusted) and adjusted models controlling for gender, city, school type, and household smoking exposure are reported. Results are presented as odds ratios (ORs) with 95% confidence intervals (CIs). All quantitative analyses were conducted using SPSS 25.0.

Qualitative analysis

We employed thematic analysis following Braun and Clarke’s six-phase framework¹⁸. Two researchers independently familiarized themselves with the data by repeatedly reading transcripts. Initial codes were generated systematically to identify features relevant to perceptions of message effectiveness. Codes were then collated into potential themes, which were reviewed against coded extracts and the entire dataset. Themes were defined and named through iterative refinement, and finally synthesized into a coherent narrative. Qualitative data were managed using Nvivo 12.0 software.

Triangulation

Triangulation was achieved by comparing quantitative effectiveness rankings with qualitative explanations of message characteristics, ensuring convergent validation of findings across methodological approaches.

Participant characteristics

A total of 172 participants were initially recruited, and 147 valid questionnaires were obtained after excluding incomplete responses. The final sample comprised 147 participants with a mean age of 14.6 years (SD=2.7; range 11–23 years). The sample was balanced by gender (51.0% male, 49.0% female) and relatively balanced by city (55.1% from Beijing, 44.9% from Kunming). School types were represented as follows: primary school (23.8%), middle school (27.9%), high school (33.3%), and university (15.0%). The vast majority of participants were never smokers (98.6%), with only 2 current smokers (1.4%) and 3 current electronic cigarette users (2.0%). Regarding household smoking exposure, 53.1% of participants reported living with someone who smokes, and 32.7% reported that indoor smoking was permitted in their homes. These characteristics are summarized in Table 1.

Overall message effectiveness rankings

Table 2 presents the rankings of perceived effectiveness. Messages providing specific information about severe, long-term health consequences were associated with the highest rankings. The message warning about multiple types of cancer (M4: ‘Smoking causes multiple cancers, including lung cancer, oral cancer, laryngeal cancer, and acute leukemia’) ranked first with an EI of 0.285, with 29.9% of participants selecting it as most effective and only 1.4% selecting it as least effective. The message about severe lung disease and COPD (M2) ranked second, with an EI of 0.232, and was selected as most effective by 25.9% and as least effective by 2.7%.

In contrast, several messages showed limited or negative effectiveness. The message ‘No smoking, I’m healthy, I’m fashionable’ (M9) ranked last with an EI of -0.470, with nearly half of all participants (49.7%) rating this message as least effective. In comparison, only 2.7% rated it as most effective. The message about periodontitis (M3) also showed low effectiveness (EI= -0.204), with 21.1% rating it as the least effective.

The widely used message ‘Refuse the first cigarette’ (M8) achieved an EI of -0.008. This near-zero score reflects the message’s polarizing nature: 14.2% of participants rated it most effective while 15.0% rated it least effective, resulting in no net positive effect. The complete rankings are presented in Table 2.

Binary logistic regression analyses

Binary logistic regression analyses examined whether demographic factors independently predicted message selection (Table 3). For the most effective message, the adjusted model showed that female participants had higher odds of selecting the cancer message (M4) as most effective compared to males (AOR=2.31; 95% CI: 1.15–4.64, p=0.019), after controlling for city, school type, and household smoking exposure. City (Kunming vs Beijing: AOR=1.89; 95% CI: 0.94–3.81, p=0.074) and school type were not significantly associated with selection of M4 in the adjusted model.

For the least effective message, the adjusted model confirmed that gender was significantly associated with selecting M9 (fashionable message) as the least effective. Female participants had markedly higher odds of selecting M9 as least effective (AOR=3.85; 95% CI: 1.97–7.52, p<0.001). City and school type were not significantly associated with M9 selection in the adjusted model. Household smoking exposure (living with a smoker) was also not a significant predictor in either model.

Subgroup analyses

Chi-squared tests examined associations between demographic variables and message selection. As shown in Table 4, gender was significantly associated with perceptions of the least effective messages (p<0.001). Other comparisons, including city and school type, did not reach statistical significance.

As shown in Table 4, the significant gender difference in least effective message selection (p<0.001) was characterized by different patterns of message selection for two messages. Among females, 66.7% rated the ‘fashionable’ message (M9) as least effective (EI= -0.653), compared with 33.3% of males (EI= -0.293). Conversely, 34.7% of males rated the periodontitis message (M3) as least effective (EI= -0.333), compared with 6.9% of females (EI= -0.069). Regarding the most effective messages, 38.9% of females selected the cancer message (M4), compared with 21.3% of males; 17.3% of males selected the directive message ‘Refuse the first cigarette’ (M8), compared with 11.1% of females.

Qualitative findings

Qualitative analysis of 16 focus group discussions identified characteristics of effective and ineffective tobacco control messages from two aspects: ‘theme and message characteristics’ and ‘presentation style’. Effective characteristics included the presentation of specific health hazards, the severity and irreversibility of consequences, and factual grounding. Ineffective characteristics included: non-specific harm presentation, directive tones, abstract language, and unfamiliar health conditions.

Theme 1: Severity and irreversibility of health consequences

Participants consistently expressed that messages describing severe, life-threatening health outcomes were most impactful. Cancer and lung disease messages resonated strongly because participants perceived these conditions as catastrophic, irreversible, and truly frightening. Representative quotes included:

In contrast, conditions perceived as less severe or more treatable failed to generate strong deterrent responses:

Theme 2: Specificity and factual grounding

Participants valued messages that provided concrete, specific information about health consequences.

Messages listing multiple cancers or describing specific disease mechanisms were perceived as more credible and informative than vague warnings:

Conversely, abstract or general statements were dismissed as lacking substance:

Theme 3: Credibility and authenticity

Young people demonstrated sophisticated evaluation of message credibility. Messages perceived as scientifically grounded and factually accurate were viewed favorably, while those seen as exaggerated, patronizing, or disconnected from reality were rejected. The ‘fashionable’ message was particularly criticized:

Scientific information about addiction mechanisms was viewed as credible:

Theme 4: Autonomy and tone

Reactions to directive messages revealed sensitivity to perceived threats to autonomy. The ‘Refuse the first cigarette’ message polarized participants: some appreciated its directness, viewing it as protective guidance, while others perceived its imperative tone as controlling:

This polarization is consistent with the message’s near-zero effectiveness index.

Triangulation of quantitative and qualitative evidence

The integration of quantitative effectiveness rankings with qualitative explanatory insights achieved triangulation that strengthens confidence in the research findings. Quantitative data revealed patterns in message effectiveness, with health-consequence messages focusing on cancer and lung disease ranking highest. Qualitative analysis provided convergent validation by identifying the specific characteristics that young people found compelling in these same messages, including severity, specificity, and factual grounding.

This triangulation was particularly valuable for understanding why certain messages failed to achieve their intended effect. The quantitative data showed poor performance for the ‘fashionable’ message (EI= -0.470) and the directive ‘Refuse the first cigarette’ message (EI= -0.008), while qualitative insights revealed the underlying mechanisms including perceptions of inauthenticity, lack of substantive information, and inappropriate directive tone.

This mixed-methods study provides preliminary findings regarding which text-based tobacco control message strategies may be more effective among Chinese young people and, importantly, why they may be effective. The findings offer initial guidance for campaign developers while contributing to theoretical understanding of message processing among young people.

The limited effectiveness of directive messaging

Perhaps the most notable finding of this study is the near-zero effectiveness of the ‘Refuse the first cigarette’ message (EI= -0.008). This widely used message, representative of traditional directive approaches to tobacco prevention, achieved no net positive effect among Chinese young people. The message’s polarizing nature – while some participants appreciated its directness, others reported reactions consistent with patterns described in Psychological Reactance Theory⁶ – has implications for current campaign practices in China, where this message remains prominently featured in school-based tobacco prevention programs. However, we acknowledge that the cross-sectional data and small sample size preclude definitive causal attribution of this polarization to reactance mechanisms. Alternative explanations, such as familiarity effects or perceived message relevance, may also contribute to the observed pattern.

The patterns observed among Chinese young people are noteworthy, given traditional cultural values emphasizing respect for authority. Our data suggest that contemporary Chinese young people, like their Western counterparts, may resist messages perceived as threatening their autonomy^7,8. From a campaign effectiveness perspective, messages that may alienate substantial portions of the target audience warrant further evaluation compared to messages achieving broader consensus, such as those emphasizing severe health consequences.

The effectiveness of specific health consequence messages

The preference for messages detailing specific, severe health consequences aligns with the threat appraisal component of the Extended Parallel Process Model (EPPM)⁵. Messages about cancer and severe lung disease may have succeeded because they established high perceived severity – outcomes universally recognized as catastrophic and life-threatening. Qualitative findings revealed that young people engaged in sophisticated evaluation of threat severity, assessing not just whether an outcome was negative, but whether it was severe enough to merit serious concern and whether it was reversible. This finding extends previous research^17,19 by suggesting that among Chinese young people, the threshold for perceived severity sufficient to motivate deterrence may be quite high.

The failure of aspirational messaging

The ‘No smoking, I’m healthy, I’m fashionable’ message (EI= -0.470) demonstrated the limited effectiveness of aspirational messaging among Chinese young people. This approach attempted to associate non-smoking with desirable identity characteristics but was broadly rejected. Qualitative data revealed multiple reasons for rejection: young people did not find the message credible, considered it patronizing, and criticized its lack of factual information. The significant gender difference in rejection rates (females: EI= -0.653 vs males: EI= -0.293; p<0.001) suggests that females may be particularly sensitive to perceived inauthenticity in health messaging²⁰.

Strengths and limitations

This study has several strengths. The mixed-methods design enabled both quantitative comparison across messages and qualitative exploration of underlying mechanisms. The sample spanned a wide age range (ages 11–23 years), consistent with the WHO definition of young people (ages 10–24 years), and included diverse educational levels. The inclusion of two cities with different tobacco control characteristics increased geographical diversity. The triangulation of quantitative and qualitative findings provided validation of the identified effective message characteristics.

However, several limitations warrant consideration. First, the modest sample size (n=147) limits the statistical power for subgroup analyses and the generalizability of findings. Second, the sample was drawn from only two cities in China (Beijing and Kunming) and does not capture the full range of cultural, economic, and social contexts across China’s diverse regions. As such, comparisons with international studies should be made with caution, and findings may not be generalizable to young people in other Chinese cities or provinces. Third, the sample’s predominantly never smoker composition (98.6%) means findings reflect perceptions of low-risk youth, though this aligns with our focus on smoking initiation prevention. Fourth, the cross-sectional design of the quantitative component precludes causal inferences regarding the relationship between message characteristics and effectiveness perceptions. Fifth, participants’ responses may have been influenced by social desirability bias, particularly given the classroom setting and the socially normative nature of anti-smoking attitudes among young people. Sixth, socio-economic status was not measured, which may confound message perceptions and limit the generalizability of findings. Seventh, we measured perceived effectiveness rather than actual behavioral impact, and perceived effectiveness may not directly translate to behavioral change or actual reductions in smoking initiation. Eighth, the effectiveness index (EI) is a descriptive metric that does not account for distributional uncertainty or enable formal statistical comparison across all messages simultaneously. Ninth, the exclusive focus on text-based messages limits ecological validity, as contemporary tobacco prevention messaging increasingly employs visual, multimedia, and social media formats. Future research should examine how the message characteristics identified in this study perform when integrated with visual and digital media formats that are more representative of young people’s media environments.

Implications

These preliminary findings suggest that campaign resources may benefit from prioritizing messages that clearly communicate serious health risks, particularly cancer and lung disease. Messages should provide concrete details rather than vague warnings. Messages relying on abstract concepts without factual support, such as the ‘fashionable’ message, may warrant reconsideration. Regarding the widely used directive message ‘Refuse the first cigarette’, these exploratory findings suggest it may warrant further evaluation using larger, more representative samples before its continued use in national campaigns can be fully supported. Rather than recommending immediate discontinuation, we suggest that campaign developers consider supplementing existing messages with those emphasizing specific, severe health consequences, which showed more consistent positive perceptions in this study.