The quantification of unobservable psychological, behavioral, and epidemiological constructs is a foundational endeavor across the social, behavioral, and clinical health sciences. Because latent variables\u2014such as psychological distress, perceived stigma, food insecurity, and health-related self-efficacy\u2014cannot be measured directly via physical instrumentation, investigators rely on psychometric scales composed of observable multi-item indicators. However, the development of psychometrically sound measurement tools is an intricate, multi-stage scientific endeavor that requires methodological rigor across theoretical, statistical, and operational dimensions. The framework articulated by Godfred O. Boateng, Torsten B. Neilands, Edward A. Frongillo, Hugo R. Melgar-Quiñonez, and Sera L. Young (2018) provides a comprehensive, nine-step primer systematically organized into three distinct developmental phases: Item Development<\/em>, Scale Development<\/em>, and Scale Evaluation<\/em>.<\/p>\n In Phase I (Item Development), researchers identify the target construct’s theoretical boundaries, generate a preliminary item pool through deductive and inductive modalities, and evaluate content validity using expert adjudication and cognitive pre-testing interviews. In Phase II (Scale Development), the instrument is pre-tested on target populations, subjected to item reduction strategies (including item discrimination metrics, inter-item correlations, and classical test theory thresholds), and analyzed via factor extraction techniques\u2014primarily exploratory factor analysis<\/a> (EFA) or Rasch and Item Response Theory (IRT) models\u2014to ascertain structural dimensionality. In Phase III (Scale Evaluation), the latent dimensionality is independently replicated through confirmatory factor analysis<\/a> (CFA), structural equation modeling, and measurement invariance evaluations across diverse demographic strata. Finally, psychometric adequacy is determined via composite reliability indices (Cronbach’s alpha, McDonald’s omega, ordinal theta) and empirical validity appraisals, encompassing convergent, discriminant, and criterion-related validity testing. By delineating a systematic, evidence-based roadmap, this primer safeguards against widespread measurement error, curbs construct contamination, and elevates the reproducibility and clinical utility of psychometric scales worldwide.<\/p>\n Scale Development, Psychometrics, Construct Validity, Measurement Invariance, Exploratory Factor Analysis, Confirmatory Factor Analysis, Classical Test Theory, Item Response Theory, Reliability, Latent Constructs<\/p>\n The foundational primer on scale development and validation was authored by a multidisciplinary team of methodological experts, epidemiologists, and behavioral scientists:<\/p>\n The primary purpose of the methodological framework established by Boateng et al. (2018) is to demystify, systematize, and standardize the complex processes governing the construction, refinement, and psychometric evaluation of self-report instruments. Within behavioral medicine, clinical psychology, public health, and social epidemiology, investigators routinely encounter emergent, nuanced phenomena\u2014such as household water insecurity, vaccine hesitancy, patient empowerment, and intersectional stigma\u2014that lack standardized measurement tools. In the absence of formal graduate-level training in advanced psychometric theory, researchers frequently resort to ad hoc, unvalidated questionnaires or execute fragmented analytical procedures that fail to establish true measurement integrity. Such poorly calibrated instruments inject substantial systematic and random measurement error into empirical research, attenuation of effect sizes, false-negative discoveries, and compromised patient care decisions.<\/p>\n To counteract the proliferation of subpar measurement tools, this framework operationalizes psychometrics into a structured, step-by-step roadmap. Clinically, the protocol ensures that patient-reported outcome measures (PROMs) capture genuine physiological, behavioral, and affective changes rather than semantic ambiguities or differential item functioning. Methodologically, it bridges the historical divide between traditional Classical Test Theory<\/a> (CTT) and contemporary latent variable modeling paradigms, such as Item Response Theory<\/a> (IRT) and structural equation modeling (SEM). By codifying explicit thresholds for item retention, structural evaluation, and cross-cultural invariance, the primer serves as an authoritative guide for investigators seeking to build novel diagnostic tools from theoretical inception or critically evaluate and adapt existing global health instruments.<\/p>\n In psychometric measurement theory, a latent construct<\/em> is defined as an unobservable theoretical abstraction representing an individual’s cognitive, affective, behavioral, or experiential state. Because latent variables cannot be directly registered via physical sensory apparatus, they must be inferred through a constellation of observable indicators (i.e., manifest scale items). The Boateng et al. framework conceptualizes construct formulation through three essential methodological phases containing nine sequential steps:<\/p>\n The Boateng et al. primer is grounded in the convergence of two dominant measurement frameworks: Classical Test Theory (CTT)<\/strong> and Modern Latent Variable Modeling<\/strong> (comprising Item Response Theory and Structural Equation Modeling).<\/p>\n Classical Test Theory, historically conceptualized by Spearman and formalized by Novick, Lord, and Nunnally, operates on the foundational assumption that an observed test score ($X$<\/span>) is a linear function of an unobservable true score ($T$<\/span>) and an unsystematic random error component ($E$<\/span>):<\/p>\n X = T + E<\/em><\/strong><\/span><\/p>\n Under CTT axioms, random errors are assumed to have an expected value of zero, be uncorrelated with true scores, and possess mutually uncorrelated variance across parallel test administrations. CTT provides the mathematical rationale for multi-item aggregations: by summing or averaging multiple observed indicators tapping the same latent domain, idiosyncratic item-level random measurement error cancels out, causing composite scores to approach the respondent’s true score level with greater precision.<\/p>\n Concurrently, the primer integrates Modern Latent Variable Theory, which shifts analytical focus from total test scores to the probabilistic interaction between an individual’s underlying trait level ($\theta$) and the mathematical properties of individual items. Within this paradigm, Item Response Theory (IRT)<\/a> and Rasch modeling provide item difficulty, item discrimination, and item information curves that remain theoretically sample-independent. Furthermore, the structural framework incorporates factor analytic and structural equation modeling principles articulated by Jöreskog, Thurstone, and Bentler. Latent variables are modeled as common factors responsible for the shared variance among observed indicators, separating common construct variance from unique, item-specific variance and measurement error. Consequently, the primer synthesizes CTT’s practical operational utility with the analytical sophistication of confirmatory latent variable structures.<\/p>\n Within the Boateng et al. architecture, validity is conceptualized not as a static property inherent to an instrument, but as an evolving, evidence-based argument supporting the interpretations and clinical inferences drawn from scale scores, aligning with Messick’s unified validity framework<\/a>. The primer delineates explicit empirical and judgmental procedures to document validity across multiple strata:<\/p>\n Content validity is established early in Phase I through structured quantitative protocols. Subject matter panels evaluate each proposed item using Likert-type ordinal ratings regarding relevance, clarity, and essentiality. Lawshe’s Content Validity Ratio (CVR) is calculated using the formula:<\/p>\n CVR = (ne<\/sub> – N\/2) \/ (N\/2)<\/em><\/strong><\/span><\/p>\n where ne<\/sub><\/em> represents the number of panelists rating the item as “essential,” and N<\/em> is the total panel size. Items failing to achieve statistical significance based on minimum Lawshe cutoffs (e.g., CVR ≥ 0.62 for 10 panelists) are pruned. Concurrently, Lynn’s Item-Level Content Validity Index (I-CVI) requires an agreement proportion ≥ 0.78 across multiple experts, while the Scale-Level Content Validity Index (S-CVI\/Ave) must achieve ≥ 0.90 to confirm domain comprehensiveness<\/span>.<\/p>\n Construct validity evaluates whether scale scores behave in theoretical alignment with established scientific paradigms. Boateng et al. mandate empirical verification through:<\/p>\n Criterion-related validity reflects how accurately scale scores predict an external behavioral, physiological, or diagnostic benchmark. In concurrent validity testing, the instrument is correlated with an established gold standard measured concurrently. In predictive validity testing, longitudinal designs are employed to determine whether baseline scale scores successfully forecast prospective outcomes (e.g., baseline water insecurity predicting subsequent child diarrheal incidence or linear growth faltering at 12-month follow-up).<\/p>\n Reliability represents the ratio of true score variance to observed score variance, quantifying the extent to which an instrument produces consistent, reproducible measurements free from unsystematic measurement error. Boateng et al. emphasize that while high reliability is a prerequisite for validity, reliability alone does not guarantee construct validity. The primer outlines rigorous methodological criteria for evaluating multiple reliability facets:<\/p>\n Internal consistency appraises the degree of inter-relatedness among items composing a subscale. Historically, Cronbach’s alpha ($\u0007lpha$<\/span>)<\/strong> has been the standard metric:<\/p>\n α = [k \/ (k – 1)] × [1 – (∑ σi<\/sub>2<\/sup>) \/ σt<\/sub>2<\/sup>]<\/em><\/strong><\/span><\/p>\n The primer highlights that alpha relies on stringent, frequently violated statistical assumptions: essential tau-equivalence (equal factor loadings across all items) and uncorrelated item error variances. When tau-equivalence is violated, Cronbach’s alpha substantially underestimates true scale reliability. Consequently, Boateng et al. advise computing McDonald’s omega ($\\omega$<\/span>)<\/strong> (composite reliability) derived from confirmatory factor analytic loadings:<\/p>\n ω = (∑ λi<\/sub>)2<\/sup> \/ [(∑ λi<\/sub>)2<\/sup> + ∑ θii<\/sub>]<\/em><\/strong><\/span><\/p>\n where $\\lambda_i$<\/span> denotes standardized factor loadings and $\theta_{ii}$ represents unique item error variances. In addition, when items utilize ordinal Likert formats characterized by skewness, the primer recommends calculating ordinal alpha or ordinal theta based on polychoric correlation matrices. Benchmark thresholds recommended across the literature dictate:<\/p>\n For constructs conceptualized as stable psychological traits (e.g., generalized self-efficacy or locus of control), the primer mandates the evaluation of test-retest reliability across repeated administrations. The time interval must be carefully calibrated\u2014typically between 2 and 4 weeks\u2014to prevent memory recall artifacts while minimizing genuine developmental trait maturation. The framework cautions against using simple bivariate Pearson correlations, which only measure linear association while ignoring systematic mean shifts; instead, researchers are directed to employ two-way mixed-effects Intraclass Correlation Coefficients (ICC)<\/strong> modeling absolute agreement, where an ICC ≥ 0.75 reflects satisfactory temporal stability<\/span>.<\/p>\n Factor analysis represents the mathematical core of scale development, enabling researchers to explore and confirm the dimensional architecture of an item pool. The framework articulates a dual-stage analytical workflow dividing exploratory and confirmatory strategies across distinct, independent participant samples.<\/p>\n EFA is executed during Phase II to determine the number of latent factors underlying the preliminary item set. Rather than relying on outdated rules of thumb like the Kaiser criterion (eigenvalues > 1.0, which frequently over-factors), the primer advises triangulating multiple criteria<\/span>:<\/p>\n For factor extraction, robust maximum likelihood (ML) or principal axis factoring (PAF) is recommended over principal components analysis (PCA), as PCA assumes zero measurement error and inflates factor loadings. Oblique rotations (e.g., Promax, Direct Quartimin) should be chosen over orthogonal rotations (e.g., Varimax) because latent behavioral and psychological constructs are rarely completely independent ($r = 0$<\/span>). Items displaying standardized factor loadings < 0.30 to 0.40, or complex cross-loadings differing by less than 0.15 across factors, are systematically flagged for deletion<\/span>.<\/p>\n In Phase III, CFA is conducted on an independent validation sample to formally test whether the theoretical factor model accurately reproduces the observed covariance matrix. Model adequacy is evaluated using standardized global fit indices:<\/p>\n To establish that a newly developed scale functions equivalently across diverse cultural, demographic, or clinical subpopulations, the framework prescribes multigroup confirmatory factor analysis (MGCFA). Investigators sequentially test hierarchical levels of invariance: Configural Invariance<\/em> (identical factor structure across groups), Metric Invariance<\/em> (equal factor loadings, validating cross-group structural equivalence), Scalar Invariance<\/em> (equal item intercepts, permitting direct comparison of latent factor means), and Strict Invariance<\/em> (equal unique item residual variances). Degradation of fit ($Delta\text{CFI} > 0.010$<\/span> or $Delta\text{RMSEA} > 0.015$<\/span>) indicates differential item functioning, requiring partial invariance modeling or cross-cultural recalibration.<\/p>\n The operational characteristics of measurement instruments developed under the Boateng et al. methodological framework adhere to standardized psychometric conventions:<\/p>\n The landmark methodological primer “Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research”<\/em> was formally published in 2018<\/strong> in Frontiers in Public Health<\/em>. The publication is distributed under the terms of the Creative Commons Attribution License (CC BY 4.0)<\/a>. As an open-access methodological treatise, the nine-step framework, developmental guidelines, and conceptual workflows are freely accessible to researchers, clinicians, academic faculties, and global health practitioners worldwide without licensing fees or financial encumbrances.<\/p>\n Investigators who develop specific, standalone psychometric instruments (e.g., the Household Water Insecurity Experiences [HWISE] Scale, Coping Self-Efficacy Scales, or Breastfeeding Support Instruments) utilizing this primer retain separate proprietary or open-access copyrights over their distinct questionnaire items. While the methodological protocol itself is completely open, individual scales cited within the primer may require explicit authorization from original test developers, institutional copyright holders, or commercial publishers prior to diagnostic or clinical deployment.<\/p>\n The source document, “Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer”<\/em> (Boateng et al., 2018), is an overarching methodological guideline and psychometric blueprint rather than a single clinical questionnaire. The official, concrete measurement items derived from this methodology (such as specialized health insecurity questionnaires, behavioral scales, or clinical screening tools developed by the authors) are subject to individual author copyright, empirical field documentation, or institutional permissions, and the complete standardized item inventory is not reproduced in the open public domain.<\/p>\n To acquire the authorized, fully validated item inventories for specific empirical scales developed using this primer\u2014such as the Household Water Insecurity Experiences (HWISE) Scale or related global health measures\u2014researchers must contact the primary corresponding author (Dr. Godfred O. Boateng at Northwestern University \/ University of Texas at Arlington) or access the specific peer-reviewed study publications detailing those individual empirical instruments.<\/p>\n The core structural product generated within the Boateng et al. (2018) primer is the systematic Nine-Step Methodological Scale Development and Validation Protocol. Below is the operational evaluation framework used by psychometricians to assess the methodological fidelity of newly developed scales:<\/p>\n2. Keywords<\/h2>\n
3. Authors<\/h2>\n
\n
4. Purpose<\/h2>\n
5. Psychological Construct<\/h2>\n
Phase I: Item Development<\/h3>\n
\n
Phase II: Scale Development<\/h3>\n
\n
Phase III: Scale Evaluation<\/h3>\n
\n
6. Theoretical Framework<\/h2>\n
7. Validity<\/h2>\n
Content and Face Validity<\/h3>\n
Construct Validity: Convergent and Discriminant Validation<\/h3>\n
\n
Criterion and Predictive Validity<\/h3>\n
8. Reliability<\/h2>\n
Internal Consistency Reliability<\/h3>\n
\n
Temporal Stability: Test-Retest Reliability<\/h3>\n
9. Factor Analysis<\/h2>\n
Exploratory Factor Analysis (EFA)<\/h3>\n
\n
Confirmatory Factor Analysis (CFA) and Fit Evaluation<\/h3>\n
\n
Measurement Invariance<\/h3>\n
10. Instrument \/ Measurement Tool<\/h2>\n
\n
11. Permissions & Fee and Test Year<\/h2>\n
12. References<\/h2>\n
\n
13. Items of the Scale<\/h2>\n
Illustrative Structural Implementation: The 9-Step Methodological Audit Checklist<\/h3>\n
Phase I: Item Development Audit<\/h4>\n
\n
\n [ ] Deductive literature review executed across relevant theoretical disciplines.
\n [ ] Inductive qualitative interviews \/ focus groups conducted with target demographic.
\n [ ] Initial pool generated with over-inclusive item volume (typically 2–3× final target).
\n [ ] Response scale options (Likert, semantic differential, dichotomous) explicitly established.\n <\/div>\n<\/li>\n
\n [ ] Quantitative Lawshe’s CVR or Lynn’s CVI calculated per item.
\n [ ] Items pruned or linguistically revised based on standardized CVR\/CVI thresholds.\n <\/div>\n<\/li>\n
\n [ ] Participant comprehension, recall burden, and response mapping formally evaluated.
\n [ ] Ambiguous, double-barreled, or culturally insensitive items modified or eliminated.\n <\/div>\n<\/li>\n<\/ul>\nPhase II: Scale Development Audit<\/h4>\n
\n