If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Robert Edgren, Department of Tobacco, Gambling and Addiction, National Institute for Health and Welfare, P.O. Box 30, FI-00271, University of Helsinki, Helsinki, Finland.
Institute of Behavioural Sciences, Faculty of Behavioural Sciences, University of Helsinki, Helsinki, FinlandDepartment of Tobacco, Gambling and Addiction, National Institute for Health and Welfare, Helsinki, Finland
Department of Tobacco, Gambling and Addiction, National Institute for Health and Welfare, Helsinki, FinlandInstitute of Clinical Medicine, Faculty of Medicine, University of Helsinki, Helsinki, Finland
Finnish Office for HTA (FINOHTA) at National Institute for Health and Welfare, Helsinki, FinlandDepartment of General Practice, Institute of Public Health, University of Copenhagen, Copenhagen, Denmark
Department of Tobacco, Gambling and Addiction, National Institute for Health and Welfare, Helsinki, FinlandInstitute of Clinical Medicine, Faculty of Medicine, University of Helsinki, Helsinki, Finland
Department of Tobacco, Gambling and Addiction, National Institute for Health and Welfare, Helsinki, FinlandInstitute of Clinical Medicine, Faculty of Medicine, University of Helsinki, Helsinki, Finland
This review aims to clarify which instruments measuring at-risk and problem gambling (ARPG) among youth are reliable and valid in light of reported estimates of internal consistency, classification accuracy, and psychometric properties. A systematic search was conducted in PubMed, Medline, and PsycInfo covering the years 2009–2015. In total, 50 original research articles fulfilled the inclusion criteria: target age under 29 years, using an instrument designed for youth, and reporting a reliability estimate. Articles were evaluated with the revised Quality Assessment of Diagnostic Accuracy Studies tool. Reliability estimates were reported for five ARPG instruments. Most studies (66%) evaluated the South Oaks Gambling Screen Revised for Adolescents. The Gambling Addictive Behavior Scale for Adolescents was the only novel instrument. In general, the evaluation of instrument reliability was superficial. Despite its rare use, the Canadian Adolescent Gambling Inventory (CAGI) had a strong theoretical and methodological base. The Gambling Addictive Behavior Scale for Adolescents and the CAGI were the only instruments originally developed for youth. All studies, except the CAGI study, were population based. ARPG instruments for youth have not been rigorously evaluated yet. Further research is needed especially concerning instruments designed for clinical use.
Rigorous psychometric evaluation of instruments measuring at-risk and problem gambling in young people has not been accomplished yet. Reporting estimates of internal consistency from previous articles is not enough, considering the weak theoretical foundation of these instruments. Reliability testing in population-based studies and validation for clinical use are needed.
Adolescents have persistently been reported to have higher problem gambling rates than the adult population [
At-risk and problem gambling among Finnish youth: The examination of risky alcohol consumption, tobacco smoking, mental health and loneliness as gender-specific correlates.
]. The prevalence rates of adolescent problem gambling in relation to adult prevalence rates have raised concerns about the validity of screening instruments, notably whether adolescent rates are exaggerated [
Youth gambling: Prevalence, risk and protective factors and clinical issues.
in: Increasing The Odds: What Clinicians Need to Know About Gambling Disorders. Volume 7. National Center for Responsible Gaming (NCRG),
Beverly, MA2012: 14-25
] conducted a critical review of youth problem gambling assessment instruments, identifying four instruments. Two of these were simply adaptations of adult instruments where item wording was modified to better represent potential adolescent-specific adverse consequences [
Youth gambling: Prevalence, risk and protective factors and clinical issues.
in: Increasing The Odds: What Clinicians Need to Know About Gambling Disorders. Volume 7. National Center for Responsible Gaming (NCRG),
Beverly, MA2012: 14-25
]. These were the South Oaks Gambling Screen Revised for Adolescents (SOGS-RA) and the Diagnostic Statistical Manual IV (Multiple Response format) adapted for Juveniles (DSM-IV-J; DSM-IV-MR-J; Table 1). The Massachusetts Gambling Screen (MAGS) was developed on a sample of high school students, but according to developers, it could be used for both adolescents and adults [
Signs and symptoms of problem gambling and its negative consequences.
12 Items with two response options (yes/no) scored 0–1. Four additional items provide insight to an individual's gambling, but not used in scoring Time frame: Past year
Recent studies have preferred the narrow criteria: sum score 2–3 = at-risk gambling; ≥4 = problem gambling
•
Widespread use.
•
Confusing due to the two scoring procedures (“broad” and “narrow”)
Based on the DSM-IV criteria. The DSM-IV-MR-J is a modified version of the DSM-IV-J featuring simpler language, fewer details and multiple response options.
12 Items with two response options (yes/no) scored 0–1 Time frame: Past year
Sum score ≥4 = problem gambling
•
The DSM-IV-J is considered more conservative than the SOGS-RA.
•
It fails to measure the DSM-IV criterion “loss of control.” The exclusion criterion is claimed to be premature. The multiple-response format of the DSM-IV-MR-J is collapsed when calculating the scores.
•
Insufficient evidence of the classification accuracy of the instrument
. DSM-IV was used as a reference standard in the development process.
14 Items; seven items are scored in a scale based on item weights from a discriminant function analysis; yes/no Time frame: Past year
Each item is scored 0 for no and 1 for yes. Each item score is multiplied by a weight and then summed along with constant using a weighted scoring algorithm derived from a discriminant function analysis. The instrument classifies respondents into nonpathologic gambling, transitional gambling, or pathologic gambling
Both the MAGS-7 subscale and the DSM-IV subscale are used independently of one another within the field, which is a likely source of confusion and complicates comparability across studies.
•
All information up to year 2015 is based on three articles.
]. ARPG instruments are used to make judgements at the individual level, and for decision-making at a societal level, making it essential for such tools to be valid and reliable [
]. To our knowledge, a systematic review has not yet been conducted on this subject.
This systematic review aims to summarize existing evidence on the psychometric evaluations of instruments designed to evaluate ARPG among youth. The transition from being an adolescent to a young adult, and simultaneously becoming able to gamble legally, is a critical phase concerning gambling behavior. The age range of youth is limited to a maximum of 28 years [
]. Specifically, we wish to clarify which instruments measuring ARPG among youth are reliable and valid for both population-based and clinical studies in light of reported estimates of internal consistency, classification accuracy, and psychometric properties. Consequently, the aim was to identify suitable instruments presently available and provide insight on what branches of the field require further investigation.
Methods
Search strategy
A structured electronic search was conducted (November 2014) in PubMed, Medline, and PsycInfo databases, covering articles published between 2009 and 2014. The search result was processed and reported according to methods recommended in the PRISMA statement [
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration.
O'Connor D, Green S, Higgins JPT. Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. Available at: http://handbook.cochrane.org/chapter_5/5_defining_the_review_question_and_developing_criteria_for.htm. Accessed March 31, 2016.
]: P (patient, i.e., population), I (intervention, i.e., instrument), C (comparator, i.e., reference instrument), and O (outcome, i.e., reliability). The search strategy for PubMed is in the Supplementary Data; further search strategy details across bibliographic databases are available on request. On completion, the searches from each database were documented and references imported into RefWorks, where duplicates were eliminated.
Reference lists of the included articles and of identified review articles were scrutinized to find articles unrecognized in initial searches, resulting in eight additional articles for the systematic review. Database searches were updated in November 2015. The flow chart of the article selection process is presented in Figure 1.
Figure 1Flow chart of the retrieval and inclusion/exclusion process for articles used in review.
Original research articles written in English were accepted. Publications in peer-reviewed journals, doctoral theses, and institutional reports were accepted. Both population-based samples and clinical samples were eligible. First, 822 abstracts were evaluated using the following exclusion criteria: (1) non–gambling-related research topic; (2) sample age >28 years; (3) no gambling instrument employed; (4) case study, commentary, editorial, or letter; and (5) other (specification required; for example, papers involving only families of young gamblers or youth whose parents gambled were excluded).
Second, 445 full texts of the articles, doctoral theses [
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
] (later referred as articles) were evaluated using inclusion criteria matching the PICO:
P: ≤28 years of age;
I: instrument designed to assess youth gambling; and
O: instrument reliability reported.
A comparator criterion was not required for inclusion but was included for labeling the articles. It referred to whether the study used a reference standard in addition to the primary instrument for gambling assessment. Six articles repeating the same reliability estimates as the primary source were excluded. Altogether, 50 articles were included.
In all phases, two researchers independently assessed all articles. The joint probability of agreement was 90.9% for the exclusion of articles based on the abstracts. The joint probability of agreement for the inclusion of articles based on full-text evaluation was 92.4%. Disagreement in evaluation was resolved by discussion and with a third independent researcher.
Quality assessment
The articles were appraised using the revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) [
], which assesses the risk of bias and the applicability of articles. QUADAS-2 questions were tailored for our review, tested on a subsample of articles and modified to ensure unambiguous assessment between researchers. An important change was omitting the evaluation of risk of bias regarding the reference standard. This decision was made because both the index test and reference standard (if measured) were applied identically within the gambling context. Information pertaining to the reference standard was inferred from the applicability assessment. The risk of bias was assessed by five questions (Table 2), and each guiding question required an answer of “high,” “low,” or “unclear” risk of bias, where unclear risk refers to suboptimal reporting. The applicability of articles was evaluated with three questions (Table 2), and each item was scored as either “good,” “poor,” or “unclear” applicability. The joint probability of agreement was 89.8%. Disagreement was resolved by discussion (see Eligibility criteria section).
Table 2Quality assessment of included articles using the revised Quality Assessment of Diagnostic Accuracy Studies tool
Information from the articles was compiled into tables (Tables 3 and 4). Data from external articles (e.g., methodological reports) were used when necessary to complete study information.
Table 3Characteristics, applicability, and risk of bias of included studies on instruments for measurement of gambling problems among young people
a) Design, b) sampling method, c) administration method, d) statistical methods related to reliability and validity, e) references of ARPG: comparator measurement.
Sample size in analysis and in total, gender distribution from analyzed sample (unless otherwise specified as total sample), mean age (range) in years.
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
Two community samples; Sample A: all French-speaking school boys in economically disadvantaged area; sample B: representative sample of kindergarteners
c)
Self-report questionnaires
d)
Cronbach alpha
e)
Correlation with gambling problems at age 16 (SOGS-RA) and 23 (SOGS).
n = sample A: 502 (1,037); sample B: 663 (2,000) female: 0% age: middle adolescence 16.2/16.2; early adulthood 22.8/22.5
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
Variety of gambling activities from adolescence to age 30 and association with gambling problems: A 15-year longitudinal study of a general population sample.
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
Gambling Addictive Behavior Scale for Adolescents (no classifications specified)
a)
Cross-sectional
b)
Convenience stratified school sample classes selected randomly
c)
Self-report questionnaire
d)
Cronbach alpha, principal component analysis, ROC analysis
e)
Correlation with other variables
n = 299 (320) female: 40.8% age: not specified
Good Good Good
Low Unclear Low
CAGI = Canadian Adolescent Gambling Inventory; CRAGS = Clinician Rating of Adolescent's Gambling Severity; DIS-IV = Diagnostic Interview Schedule, Version IV; DSM-IV-J/DSM-IV-(MR)-J = the Diagnostic Statistical Manual IV (Multiple Response format) adapted for Juveniles; GPSS = Gambling Problem Severity Subscale; GRCS = Gambling-Related Cognitions Scale; MAGS = Massachusetts Gambling Screen; MSOGST = Modified South Oaks Gambling Screen for Teens; NL-CLiP = short screen for problem gambling among children based on criteria identified in NODS-CLiP (NODS-CLiP: Diagnostic Screening for Gambling Disorders [NODS] Loss of Control, Lying and Preoccupation [CLiP]); ROC = receiver operating characteristic; SOGS-RA = the South Oaks Gambling Screen–Revised for Adolescents.
a a) Design, b) sampling method, c) administration method, d) statistical methods related to reliability and validity, e) references of ARPG: comparator measurement.
b Sample size in analysis and in total, gender distribution from analyzed sample (unless otherwise specified as total sample), mean age (range) in years.
c QUADAS-2 assessment for applicability concerns: patient selection; index test; reference standard.
d QUADAS-2 assessment of risk of bias concerning patient selection, index test, and flow and timing.
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
SOGS-RA α = .78 (sample A) and α = .78 (sample B); correlation with SOGS-RA and SOGS r = .22 (p < .05) and .28 (p < .05); SOGS and SOGS-RA were metrically invariant, and thus psychologically comparable.
At-risk and problem gambling displayed stability during 2004 and 2005 (preintervention), with a significant decrease after the removal bank note acceptors in 2006 (postintervention). Thus, the SOGS-RA displayed stability and sensitivity to change. No gender- or age-related differences were evident in the reduction of gambling problem prevalence (i.e., 2005–2006).
Chinese version of the Gamblers Belief Questionnaire α = .91; Chinese version of the Gamblers Belief Questionnaire correlation range with other scales .40–.75
Factor loadings range .53–.83 (p < .001), CFI = .96; TLI = .97; RMSEA = .03; Suggests single factor structure for SOGS-RA. Items “feeling bad about money lost” and “gambling more than planned” had the highest endorsement rate. Absence from school due to betting, borrowing money, and stealing for betting were the most discriminative. Items “lying about winning” and “wanting to stop gambling” had the lowest discrimination. Majority of items had good discrimination. Screen accurately measures medium to high levels of problem gambling (i.e., item severity located along intended range).
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
Multiple correspondence analysis principle component (eigenvalue = 3.875) explained 32.3% of variance; test–retest κ coefficient range = .53–.80; α (males) = .786; α (females) = .707; Suggested single factor structure for SOGS-RA. The least endorsed items were “absent from school due to betting” and “borrowed or stolen for bets or debts.” The most endorsed items were “gambling more than intended” and “feeling bad about the amount bet.” The SOGS-RA score was positively associated to gambling frequency. No gender differences were evident in item endorsements. The SOGS-RA seems to be stable over time.
SOGS-RA α = .94; DSM-IV-J α = .93; SOGS-RA and DSM-IV-J correlation r = .64; DSM-IV-J is a more conservative instrument for measuring pathologic gambling than SOGS-RA.
SOGS-RA α = .73; Gambling Attitude Scale α = .80; Gambling Attitude Scale discriminated nonproblem gamblers (more cautious perception) from at-risk and problem gamblers. Problem gamblers scored higher on the items from the profitability factor than at-risk and nonproblem gamblers.
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
SOGS-RA (6 items) α = .70; Most endorsed items in 2013 were “gambling more than planned” (1.7%) and “experiencing problems with family or school due to gambling” (1.5%). Females were significantly less likely than males to endorse the previously mentioned items (p < .05).
SOGS-RA (six items) α = .71, AUC = .80 (concordance of short version and full version of SOGS-RA); The short version of the SOGS-RA may overestimate prevalence rates.
McNemar χ2 (1, N = 88) = 8.77, p ˂ .05; At baseline the prevalence of ARPG (measured with SOGS-RA) in the training group was 41%. After training (approximately 6 months after pretest) a significant and medium in size reduction in the prevalence of ARPG was evident (prevalence at follow up: 28%). The intervention intended to increase correct knowledge about gambling, reduce misconceptions, economic optimistic view of gambling profitability, and superstitious beliefs. The intervention contained activities, PowerPoint slides, a video, and collective discussions.
Variety of gambling activities from adolescence to age 30 and association with gambling problems: A 15-year longitudinal study of a general population sample.
SOGS-RA α = .67; omega = .68; Asian-Americans were more likely to endorse the following items compared to white Americans: lying about winning; gambling more than intended; felt bad about money bet; hidden any signs of gambling.
DSM α = .80; SOGS-RA α = .75; DSM-IV-MR-J and SOGS-RA correlation r = .892 (p < .001); SOGS-RA κ = .833 (p < .001); SOGS-RA is more liberal in classifying gambling pathology. SOGS-RA classified 34 of 35 pathologic gamblers correctly, using the DSM-IV-MR-J as the reference. DSM distinguished between social, at-risk, and pathologic gamblers. DSM item on “Escape” identified as best discriminator. Those who gambled at least 1 per week scored significantly higher than participants who gambled more seldom. SOGS-RA sensitivity = .97 (34/35), specificity .986, false-positive rate = .20 and false-negative rate = .00015. DSM pathologic gamblers were likely to spend more money on gambling than nonpathologic gamblers.
Nonproblem/problem κ = .633 (only gamblers) and .778 (gamblers and nongamblers); There was a lack of consistency in responses of comparable questions in the two instruments; NL-CLiP is accurate in classifying nonproblematic and problematic gamblers, but not in distinguishing between at-risk and problem gamblers (DSM-IV-MR-J as reference)
DSM-IV-J α = .90; GRCS α = .97 (subscale α range .77–.91); Average correlation with five-factor model of GRCS = .82; At-risk and problem gamblers scored significantly higher than nonproblem gamblers on the entire GRCS scale and all its subscales separately with significant gender interaction evident throughout analyses (males scoring higher); 32% of the variance in DSM-IV-J was explained by the GRCS with hierarchical multiple regression. Inability to stop gambling, illusion of control and gambling related expectancies subscales were significant unique predictors of at-risk and problem gambling.
DSM-IV-MR-J α = .86; sensitivity of items = .22–.78; specificity of items = .94–.99; gambling often or sometimes odds ratio (95% CI) for ARPG = 5.78 (3.0–11.0); Scree plot of exploratory factor analysis supports 1-factor solution, accounting for 40.1% of variance and correlated positively with the psychological states preoccupation, tolerance, withdrawal, loss of control, escape, and chasing. Illegal acts, tolerance, loss of control, and lies were the most commonly endorsed and most sensitive items in identifying ARPG. Item on illegal acts was the least specific. Lowest sensitivity was for items on escape, risked job/education/relationship, and withdrawal.
All the MAGS items displayed different functioning between males and females. Males were more likely to endorse each item than females, given otherwise equal scores for the latent variable. This indicates that the criteria are more valid for males than females.
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
Odds ratios (95% CI): ACG = 7.66 (4.34–13.53); ANCG = 70.84 (43.41–115.62); PrG = 11.34 (3.82–33.61); Using MAGS, Latent Class Analysis indicated a four-class solution to be optimal for DSM IV and five criteria: low-risk (LG; most common), at-risk chasing gambling (ACG), at-risk negative consequences gambling (ANCG), and problem gambling (PrG; least common). Inclusion/exclusion of item on illegal acts had little effect on the classification of gambling groups. LG was characterized by low probability of endorsement for all items. ACG was characterized by elevated probability of endorsement for “win back lost money” and “gambling more money over time.” ANCG was characterized by elevated probability to endorse “losing/jeopardizing relationship or career opportunities,” “committing illegal acts,” “turning to other financial sources,” and “unsuccessful attempts to reduce or quit.” PrG was characterized by elevated probability to endorse all 10 items. Compared with LG, other gambling classes were more likely to spend more than 1 hour/week gambling (p ≤ .025).
CAGI α's for four factors = .90; .90; .83; .87; test–retest intraclass correlations = .77; .90; .83; .87; CRAGS and DSM-IV measures correlations r ≤ .89; CAGI subscale correlations with gambling involvement measures r = .14–.67; Endorsement of consequence items “stealing to gamble,” “feeling guilty about gambling behaviors,” and “gambling for longer periods than planned” were much higher for phase III sample than phase II. Principal component analysis suggested four-factor solution (psychological consequences; social consequences; financial consequences; loss of control) explaining 67.3% of variance, with a balanced weight among factors. Factor correlation between .62 and .69. Confirmatory factor analysis suggests reasonably good model fit. High congruency between classifications of gold standards (DSM-IV self-rated and clinical interview, and CRAGS). Discriminant function analysis and ROC analysis revealed nine-item subscale (GPSS) to be optimal for classification performance. Measures of cognitive distortions, decision-making, and self-efficacy correlated below r = .30 with CAGI subscales. Strongest correlates related to convergent validity of problem gambling were risk taking and self-control (all r > .30), followed by impulsivity.
Gambling Addictive Behavior Scale for Adolescents α = .94; α's for subscales = .90; .89; .88; .90; Assessment by expert panel yielded content validity index of 94.3%. Final scale composed of 25 items, loading onto four factors explaining 54.9% of variance (loss of control; life dysfunction from gambling addiction; gambling experience; social dysfunction from problem gambling). Scale categorizes individuals as nongambling (AUC = .71), nonproblem gambling (AUC = .75), and problem gambling (AUC = .74) group (p ≤ .001). Factors correlated significantly with irrational gambling beliefs, gambling behavior
, and the Addictive Personality subscale of the Eysenck Personality Questionnaire and self-control (p < .001).
ARPG = at-risk and problem gambling; AUC = Area Under ROC Curve analysis; CAGI = Canadian Adolescent Gambling Inventory; CFI = comparative fit index; CRAGS = Clinician Rating of Adolescent's Gambling Severity; DIS-IV = Diagnostic Interview Schedule, Version IV; DSM-IV-J/DSM-IV-(MR)-J = the Diagnostic Statistical Manual IV (Multiple Response format) adapted for Juveniles; GPSS = Gambling Problem Severity Subscale; GRCS = Gambling-Related Cognitions Scale; MAGS = Massachusetts Gambling Screen; MSOGST = Modified South Oaks Gambling Screen for Teens; NL-CLiP = short screen for problem gambling among children based on criteria identified in NODS-CLiP (NODS-CLiP: Diagnostic Screening for Gambling Disorders [NODS] Loss of Control, Lying and Preoccupation [CLiP]); ROC = receiver operating characteristic; SOGS-RA = the South Oaks Gambling Screen–Revised for Adolescents; RMSEA = root mean square error of approximation; TLI = Tucker-Lewis index; VGS = Victorian Gambling Screen.
Of the 50 articles, 33 dealt with the SOGS-RA and 12 with the DSM-IV-J or DSM-IV-MR-J [collectively referred to as DSM-IV-(MR)-J] as the primary ARPG instrument. Three articles evaluated the MAGS, one evaluated the CAGI, and one evaluated the Gambling Addictive Behavior Scale for Adolescents (GABSA). Four studies primarily using the SOGS-RA also used one of the two DSM instruments, and one study primarily using the DSM-IV-MR-J also used the SOGS-RA. All samples were general population based, except the study on the CAGI, which used a sample from a substance use center. About half of the articles (n = 26) used cross-sectional data from local schools, including two pairs of articles based on the same data sets. Eight articles had a longitudinal study design. Five articles had cross-sectional nationally representative samples and another nine articles used cross-sectional data from nationally representative school samples, including one pair of articles using partially the same data set. Two studies had local school samples and used experimental designs. Articles originated from 17 different countries, including 16 from the United States, 10 from Canada, 7 from Italy, 3 from Australia, and 2 from Norway and 1 article from China, England, Finland, Germany, Greece, Hong Kong, Israel, Lithuania, Malaysia, South Korea, Spain, and Switzerland.
Table 2 summarizes the risk of bias in the articles. Almost half (n = 24) of the articles had done patient selection well. Most (n = 38) had conducted the index test properly, and in nearly all articles (n = 43), participants were maintained in analyses.
The applicability of the articles to this review was more variable (Table 2). In most (n = 43) of the articles, patient selection suited our study question, whereas only some had good applicability related to evaluating the index test (n = 9) or for using a reference measurement (n = 12).
South Oaks Gambling Screen Revised for Adolescents
In articles evaluating the SOGS-RA as the primary adolescent ARPG instrument, high risk of bias was often evident for patient selection but not for other items. Although the sample was often appropriate, only a few of the SOGS-RA articles primarily investigated instrument properties or used a reference (Table 2).
Estimates of internal consistency (Cronbach α) for the SOGS-RA ranged from .60 [
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
The SOGS-RA displayed good classification accuracy, with a sensitivity of .97, specificity of .99, false-positive rate of .20, and false-negative rate of .00015 when using the DSM-IV-MR-J as a reference [
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
], whereas another study found highest rates for items “Gambling more than planned” and “Experiencing problems with family or school due to gambling” [
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
]. Items “Lying about winning,” “Gambling more than planned,” “Felt bad about money lost,” and “Hidden any signs of gambling” were more likely endorsed by Asian-Americans than by white Americans [
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
], the SOGS-RA correlated positively with the SOGS; the two were considered metrically invariant, thus psychologically comparable.
Finally, the SOGS-RA indicated sensitivity to change, as the prevalence of ARPG decreased after the removal of banknote acceptors in slot machines, compared to the two preceding baseline years during which ARPG prevalence remained stable [
Of the 12 articles evaluating the DSM-IV-(MR)-J, six had a high risk of bias concerning patient selection, while four instances of unclear risk of bias were evident. Otherwise, the risk of bias was low (Table 2). In patient selection, these studies had good applicability. Two studies investigated instrument properties, and in four cases, a reference standard was used.
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
] (M = .85). Items on illegal acts, tolerance, loss of control, and lying had the highest endorsement rates and were the most sensitive in identifying ARPG, whereas items on escape, withdrawal, and risking job, education, or relationship were the least sensitive [
Three articles concerning the MAGS had low risk of bias for patient selection, whereas two articles had low risk of bias for both the conduct of the index test and flow and timing. Two of the articles had high applicability for the index test (Table 2).
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
] were concerned with instrument psychometric properties. ARPG classification was associated to a stronger likelihood of gambling at least 1–2 hours/week compared to low-risk gamblers [
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
]. All items had different functioning between genders, such that males were more likely to endorse each item, given otherwise equal scores for the latent variable [
]. Latent class analysis suggested a four-class solution for the DSM criteria (measured with the MAGS): low-risk gambling, at-risk chasing gambling, at-risk negative consequences gambling, and problem gambling [
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
] had low risk of bias on all domains and good overall applicability. It is a comprehensive assessment of the scale beyond the scope of this systematic review, so we provide a brief summary. Cronbach α for the subscales ranged from .83 to .90. The correlation with the DSM-IV criteria and the Clinician Rating of Adolescent's Gambling Severity was ≥.89, and correlation with gambling involvement measures ranged from .14 to .67. Principal component analysis and confirmatory factor analysis suggested a four-factor solution (psychological consequences, social consequences, financial consequences, and loss of control). The nine-item Gambling Problem Severity Subscale (GPSS) was optimal for classification performance as defined by discriminant function analysis and receiver operating characteristic (ROC) analysis.
] had an unclear risk of bias for conducting the index test but otherwise low risk and overall good applicability.
The developmental report of the GABSA suggested the scale to have high internal consistency, with an overall Cronbach α of .94, and subscale α ranging from .88 to .90. The scale displayed significant (p ≤ .001) classification accuracy (area under ROC curve analysis [AUC]) of .71 for the nongambling group, .75 for the nonproblem gamblers, and .74 for the problem gamblers, using reference scores from the Addictive Personality subscale of the Eysenck Personality Questionnaire. As for the psychometric properties, the scale loaded onto four factors (loss of control, life dysfunction from gambling addiction, gambling experience, and social dysfunction from problem gambling) explaining 55% of variance. The scale further displayed convergent validity by significant positive correlations with addictive personality, irrational gambling beliefs, and gambling behavior.
Discussion
Our review aimed to clarify which instruments measuring ARPG among youth are reliable and valid in light of reported estimates of internal consistency, classification accuracy, and psychometric properties. Five ARPG instruments were examined; 3 of the 33 SOGS-RA articles and 2 of the 12 DSM-IV-(MR)-J articles investigated instrument properties. Two of the three MAGS articles looked at instrument properties. The remaining articles concerned the development of the CAGI and the GABSA.
South Oaks Gambling Screen Revised for Adolescents
Most (58%) of the SOGS-RA articles had potentially biased sampling procedures. The reviewed estimates for internal consistency were similar to earlier findings [
], although variation was evident. Estimates of correlations with the SOGS-RA and external references (e.g., gambling frequency) and the corresponding DSM-IV-(MR)-J instruments parallel previous findings [
Specific items are problematic in population-based studies because of uneven endorsement rates. Controversially, there was evidence suggesting no item endorsement differences between genders [
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
], contradicting the idea of uniqueness of youth gambling compared to adult gambling. The results on the SOGS-RA highlight the lack of a unique conceptualization of gambling problems for adolescents and are inconclusive concerning gender differences [
Almost half of the SOGS-RA articles used ethnically diverse samples, consisting African-Americans, Hispanics, Asian-Americans, and samples from Hong Kong and Malaysia. Although these studies were not primarily testing the SOGS-RA, studies with different ethnicities emerged, as previously deemed necessary [
]. Cronbach α estimates among samples with high rates of African-Americans ranged from .60 to .85, which is moderate at best, but within the same range as previous estimates. Two items endorsed more often by Asian-Americans than white Americans concerned minimizing other people's worries about the respondents gambling, suggesting a difference in attitudes toward the social aspect of gambling [
], which is considered satisfactory. Two DSM-IV-MR-J articles were primarily concerned with instrument properties and indicated that items are not equivalent in their ability to detect problem gambling, and variabilities in item endorsement rates and sensitivity are evident [
]. Gambling frequency, amount of money spent on gambling, and score on other problem gambling screening instruments were positively associated to the DSM-IV-MR-J score [
]. Corresponding items of the DSM-IV-MR-J and the NLCLiP (short screen for problem gambling among children based on the criteria identified in the NODS-CLiP [Diagnostic Screening for Gambling Disorders - Loss of Control, Lying, and Preoccupation]) had deviating endorsement rates [
]. The concept of gambling to escape a negative state of mind may be foreign to and poorly understood by young individuals, leading to false-positive responses [
] but no indices for classification accuracy of the MAGS DSM-IV subscale and no information on the MAGS subscale. The seven-item subscale of the MAGS (MAGS-7) Cronbach α was .83, and the DSM-IV subscale .87 [
]. MAGS-7 correctly classified 96% of adolescents as problem gamblers, at-risk gamblers, and nonproblem gamblers when using the DSM-IV criteria as a reference. The MAGS-7 item “Are you always able to stop gambling when you want” was the least discriminating item, and low congruence for the classification of ARPGers was evident between the MAGS-7 and SOGS-RA [
We found new information concerning the psychometric properties of the DSM-IV subscale of the MAGS, suggesting there may be different types of at-risk gamblers: some mostly chase loses, whereas others more notably experience negative consequences. This finding supports the notion of multiple trajectories leading to problem gambling, as suggested by the Pathways Model [
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
The CAGI was included to this study even though it is not a population-based instrument. During its development, the CAGI was tested with student samples and a clinical sample, all resulting in the same four-factor structure [
]. Cronbach α for the factors ranged from .83 to .90 or within the range often estimated for other instruments. The CAGI displays high congruence to self-rated DSM-IV criteria, clinician-rated DSM-IV criteria, and to the Clinician Rating of Adolescent's Gambling Severity, although the validity of these comparators is unknown [
] may inhibit its use, but the nine-item GPSS subscale is shorter and asks about frequencies of concrete gambling behaviors. A unique strength of the GPSS subscale is that items inquire about the effects of gambling on peer relationships and financial consequences in a developmentally appropriate manner [
]. However, conclusions on CAGI need to be done cautiously since data are limited to one clinical study which was not reported in a peer-reviewed journal.
] did not provide directions for scoring or classification thresholds, and the instrument lacks strong theoretical framework. The content of proposed factors is questionable, as similar items are distributed across various factors. For example, the factor named loss of control includes the item “Spend money on gambling without paying for necessary things,” whereas the social dysfunction factor includes the item “Bet money or prizes for gambling beyond my pocket money.” Likewise, items “Spend more and more time gambling,” “Do not leave a place all day to gamble,” “Have little time to play other things or do activities except for gambling,” and “It is hard to pass by a PC room, billiards room, or amusement arcade without stopping by” all load onto different factors. However, items do tap into concrete behaviors.
Theoretical base of the instruments
A theoretical framework should be a cornerstone for the instruments. In gambling research, surveys largely use adapted adult instruments, where DSM-criteria represent the gold standard. The DSM criteria were originally formulated for diagnostic purposes, not for classifying individuals in a survey [
]. Moreover, research does not recognize the unique developmental characteristics of young people. Developing the CAGI is a step in the right direction.
]. Herein, the criteria for ARPG varied. Two or more criteria were used to define ARPGers, with the exception of few studies using one or more criteria [
Variety of gambling activities from adolescence to age 30 and association with gambling problems: A 15-year longitudinal study of a general population sample.
]. Notably, there are clear differences of at-risk and problem gambling among adolescents, at-risk referring to individuals who are starting to develop a number of gambling-related problems, but do not meet the established criteria for the more severe form of gambling (i.e., gambling disorder) [
] taken into account to some degree in measurement. For example, pursuing lost money indicates loss of control and instruments included items for “chasing losses” [
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
]. It is a major drawback that theory is not better integrated with measurement of ARPG. The individual-centered perspective to addiction research accounts for individual variance in a range of risk factors (i.e., impulsivity [personality trait] and social factors) [
]. This approach may be helpful to broaden perspectives in gambling research, as the development of gambling problems is influenced by gambling related beliefs, personality traits, and the motivation to gamble [
Different aspects of gambling are not equivalent indicators of problem gambling severity, so imputation of item-weighting procedures is necessary. The adult problem gambling instrument Problem and Pathological Gambling Measure [
] enables the detection of gambling problems even with lack of insight or denial of gambling problems, by formulating items accordingly. These aspects should also be noticed in the measurement of youth ARPG.
Study limitations
Most articles had poor applicability concerning the index test and reference measurements; only few had developing ARPG instruments as their primary objective. The low quality of the reviewed literature is a result in itself, but also a limitation. Measuring internal consistency exclusively with Cronbach α may be inappropriate because the assumptions are unlikely to be met [
]. We propose that Tarkkonen rho may be a more appropriate estimate of internal consistency as it applies the general form for estimating reliability, whereas Cronbach α is a special case with restrictive assumptions [
]. Regarding bias in the articles, the overall risk was mainly low. Further studies would benefit from even more in-depth quality assessment including criteria for psychometric properties of the instruments [
American Educational Research Association, American Psychological Association and National Council on Measurement in Education Standards for educational and psychological testing.
Clinical assessment was used as a comparator only rarely. Gamblers with gambling-related problems have shown more discrepancies between self-report and actual outcomes (i.e., accuracy bias, underestimation of losses or overestimation of gains) than gamblers without gambling problems [
]. However, determining the performance of a tool compared to clinical assessment is challenging without a stronger base and defined criteria for ARPG. Without a “gold standard” test, the validation is problematic.
Nearly all articles were population based, so results cannot be generalized into the clinical context [
]. Short sensitive measures based on DSM criteria are usually used to screen at-risk individuals in the clinical setting as part of a wider clinical assessment [
]; in reality, instruments for assessing gambling are often used outside the context originally intended.
Our material included methodologically diverse studies based on, for example, respondent age, ethnicity, and method of survey administration. To gain a comprehensive understanding of adolescent ARPG, gambling needs to be studied in an ecologically valid manner. This may be achieved with ambulatory methods (i.e., smart phones) [
Cross-species approaches to pathological gambling: A review targeting sex differences, adolescent vulnerability and ecological validity of research tools.
An estimate of reliability was reported for five ARPG instruments. Most articles (66%) evaluated SOGS-RA. The GABSA was the only novel instrument developed from 2009. Generally, the evaluation of reliability and validity of the instruments was superficial. Despite a very modest publication base, the CAGI seems to have a strong theoretical and methodological base. Reviewed articles with high applicability [
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
A latent class analysis of pathological-gambling criteria among high school students: Associations with gambling, risk and health/functioning characteristics.
] advocate that screening should include measures of risk taking, self-control or impulsivity, delinquent behavior, and social risk factors. The GABSA and the CAGI were the only instruments originally developed especially for youth. Studies were entirely population based, except the one concerning the CAGI. In 2010, Volberg et al. concluded that despite the questions raised regarding the validity of the SOGS-RA and DSM-IV-MR-J, these instruments are the best tools for evaluating adolescent gambling problems while waiting for a better-validated instrument [
], although cultural validation has not been an actual aim. It is encouraged that future studies look closer into gender differences, study clinical samples, and community-based samples. Attending to these issues in ARPG research may lead to a deeper understanding of the phenomena. In recognition of the similarities across addictions, it is important that accumulating knowledge from the field is integrated into measuring ARPG.
Rigorous psychometric research of youth ARPG instruments, as recommended by earlier reviews [
], has not yet been accomplished. Thus, it is untimely to name the most suitable instrument presently available. Researchers are encouraged to test reliability in population-based studies, where samples are not derived from schools, and especially in the clinical context. Reporting alphas from previous articles is not enough, considering the weaknesses in the theoretical foundation of ARPG instruments. We hope that bringing recent work to light will nourish forthcoming studies. If researchers collectively strive to improve the accuracy of ARPG measurement, even the understanding of behavioral addictions in general may advance tremendously.
Acknowledgments
The authors thank the Information Service Designer Pirjo Vuorio, National Institute for Health and Welfare, for her substantial assistance in the management of articles used during the conduct of this systematic review. All authors contributed significantly to the planning of the work and drafting the manuscript or revising the work critically for important intellectual content. The authors of this manuscript and Pirjo Vuorio did not have any interests that might be interpreted as influencing the research. APA ethical standards were followed in the conduct of the study. The corresponding author affirms that all individuals who have contributed significantly to the preparation of this manuscript have been mentioned in the Acknowledgements. Please note that the coauthor Sari Castrén has presented the protocol of our report at the sixth International Gambling Conference in February 2016 (10.-12.2.2016), in Auckland, New Zealand.
Funding Sources
This study was financially supported by the Ministry of Social Affairs and Health, Helsinki, Finland (the §52 Appropriation of the Lotteries Act, contract STM/3189/2011).
At-risk and problem gambling among Finnish youth: The examination of risky alcohol consumption, tobacco smoking, mental health and loneliness as gender-specific correlates.
Youth gambling: Prevalence, risk and protective factors and clinical issues.
in: Increasing The Odds: What Clinicians Need to Know About Gambling Disorders. Volume 7. National Center for Responsible Gaming (NCRG),
Beverly, MA2012: 14-25 (Available at:)
The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration.
O'Connor D, Green S, Higgins JPT. Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S, eds. Cochrane handbook for systematic reviews of interventions version 5.0.0 [updated February 2008]. The Cochrane Collaboration, 2008. Available at: http://handbook.cochrane.org/chapter_5/5_defining_the_review_question_and_developing_criteria_for.htm. Accessed March 31, 2016.
Cross-lagged links among gambling, substance use, and delinquency from midadolescence to young adulthood: Additive and moderating effects of common risk factors.
Italian adolescent gambling behavior: Psychometric evaluation of the South Oaks Gambling Screen: Revised for Adolescents (SOGS-RA) among a sample of Italian students.
The mental health and well-being of Ontario students 1991-2013: Detailed OSDUHS findings (CAMH Research Document Series No. 38). Center for Addiction and Mental Health,
Toronto, ON2014
Variety of gambling activities from adolescence to age 30 and association with gambling problems: A 15-year longitudinal study of a general population sample.
00075-6&id=gr1.jpg){kind=link}