The Coaching Fidelity Scale (CFS)<\/strong> (Meadan et al., 2023<\/a>) is an empirically validated observational rating checklist designed to assess the implementation fidelity of professional coaching practices within early intervention, special education, and behavioral healthcare settings. Structured around evidence-based, triadic coaching models utilized across naturalistic developmental behavioral interventions (NDBIs<\/a>), the CFS operationalizes coaching into an 11-item checklist encompassing three sequential operational phases: (a) pre-observation<\/em>, which establishes shared intentionality through strategy identification, contextual routine selection, and procedural task analysis review; (b) observation<\/em>, which captures uninterrupted caregiver-child or practitioner-child behavioral interactions; and (c) post-observation<\/em>, which facilitates reciprocal learning through coachee reflection, targeted supportive and constructive performance feedback, and collaborative problem-solving. Additionally, the CFS accommodates an optional instructional module evaluating video-recorded feedback sessions.<\/p>\n The scale was developed and psychometrically evaluated across multiple intervention cohorts comprising professionals\u2014including board-certified behavior analysts (BCBAs<\/a>), early childhood special educators, speech-language pathologists, and research interventionists\u2014coaching adult coachees (e.g., caregivers of young children under five years old with autism spectrum disorder or related developmental disabilities, paraprofessionals, and registered behavior technicians). Psychometric evaluation confirms exceptional internal consistency (Cronbach's alpha<\/a> = .91, 95% CI [.89, .93]; average inter-item correlation = .47) and outstanding interrater reliability (intraclass correlation coefficient [ICC] > .96). Exploratory factor analysis (EFA<\/a>) substantiated an unequivocal unidimensional latent structure accounting for 77.84% of the total variance, accompanied by robust model fit indices (CFI = .981, TLI = .975, RMSEA = .065, WRMR = .0449). Known-groups validity demonstrated high test sensitivity, capturing statistically significant, large-magnitude advancements in coaching fidelity from pre-intervention baseline to post-intervention assessment (Mann\u2013Whitney W<\/em> = 327.00, p<\/em> < .001, rank-biserial correlation = .84). The CFS addresses an acute methodological gap in implementation science<\/a>, providing researchers, administrators, and clinical supervisors with a psychometrically sound, low-inference observational metric to evaluate treatment adherence and optimize coach training<\/span>.<\/p>\n Coaching Fidelity Scale, implementation fidelity, observational assessment, adult coaching practices, autism spectrum disorder, developmental disabilities, early childhood intervention, parent-mediated interventions, naturalistic developmental behavioral interventions, psychometrics.<\/p>\n The Coaching Fidelity Scale was conceptualized, developed, and psychometrically validated by an interdisciplinary team of researchers specializing in early childhood special education, behavioral consultation, and family-centered intervention methodologies:<\/p>\n The primary purpose of the Coaching Fidelity Scale (CFS) is to provide an objective, low-inference, criterion-referenced observational instrument that assesses the fidelity with which professionals deliver collaborative coaching interventions to adult learners. In contemporary pediatric and educational intervention paradigms\u2014most notably within family-centered early intervention for young children with autism spectrum disorder (ASD)<\/a> and developmental delays\u2014service delivery has fundamentally transitioned from direct clinician-to-child intervention toward a triadic consultation model. In this triadic framework, the professional serves as a coach who guides, scaffolds, and empowers primary caregivers or frontline educational staff (the coachees), who subsequently deliver naturalistic, evidence-based instructional strategies directly to the child within everyday family routines.<\/p>\n Despite the ubiquitous empirical endorsement of caregiver coaching as an active mechanism for developmental acceleration, implementation science literature has persistently highlighted a "black box" phenomenon regarding coaching delivery. Historically, researchers and clinical supervisors lacked standardized, validated measurement tools to verify whether interventionists adhered to the specified active ingredients of coaching or whether the observed child outcomes were attenuated by fidelity slippage. Without direct measurement of coaching fidelity, attributing child progress or stagnation to a given intervention model remains compromised by Type III error (evaluating an intervention that was not implemented as planned). The CFS systematically solves this methodological vulnerability by operationalizing the multi-step coaching cycle into clear behavioral descriptors that can be reliably evaluated through live observation or recorded video analysis.<\/p>\n In research contexts, the CFS serves as an indispensable integrity check across randomized controlled trials (RCTs), single-case experimental designs (SCEDs), and multisite translational studies. It allows principal investigators to quantify treatment differentiation, monitor fidelity drift over time, compute dosage-fidelity interaction matrices, and ensure that multi-tiered coaching protocols remain uniform across diverse interventionists and service delivery modalities (e.g., in-person home visits versus synchronous telepractice). Clinically, the CFS functions as a formative and summative supervisory assessment tool. Clinical directors, BCBA supervisors, and early intervention administrators utilize the scale to conduct objective baseline audits of practitioner skills, guide individualized supervision plans, track professional competency acquisition during onboarding or practicum training, and identify specific practice deficits\u2014such as superficial problem-solving or the omission of systematic reflection\u2014that require targeted scaffolding.<\/p>\n The psychological construct underlying the CFS is implementation fidelity<\/strong> within professional coaching frameworks, defined as the degree to which an interventionist delivers an evidence-based coaching methodology in complete alignment with its intended procedural architecture, theoretical mechanisms, and operational principles. In the CFS framework, fidelity is conceptualized as a multi-stage, interactive process rather than a static clinician characteristic. The instrument decomposes coaching into three interconnected behavioral phases, augmented by an optional modality-specific feature:<\/p>\n The pre-observation phase captures the coach's capacity to establish instructional alignment and mutual intentionality prior to active intervention delivery. In this preparatory phase, the coach refrains from unilateral didactic instruction and instead initiates a shared dialogue with the coachee to anchor the upcoming learning episode. The specific behavioral elements within this dimension include:<\/p>\n The observation phase assesses the coach's adherence to structured observational discipline. During this phase, the coach observes the uninterrupted natural interaction between the coachee and the child. The critical psychological parameter evaluated here is the coach's restraint from unnecessary verbal intrusion, physical overshadowing, or co-opting the caregiver's parental role. By maintaining an unobtrusive presence, the coach affords the coachee genuine psychological agency, authentic practice opportunities, and space to experience naturalistic contingencies. Concurrently, the coach systematically collects observational data, notes procedural adherence, and identifies salient interaction moments to inform the subsequent feedback phase.<\/p>\n The post-observation phase represents the communicative core of adult coaching, where experiential learning is transformed into lasting behavioral competence through structured debriefing. This phase incorporates three interdependent communicative interactions:<\/p>\n To support contemporary technological adaptations, the CFS integrates an optional evaluative unit governing video feedback. In this component, the coach and coachee jointly review recorded video footage of previous or immediate interactions. The coach pauses video segments at critical decision points to highlight nuanced nonverbal cues, analyze child communicative initiations, review micro-behaviors that are challenging to discern in real time, and prompt deeper coachee reflection through objective visual playback.<\/p>\n The Coaching Fidelity Scale is grounded at the intersection of three theoretical paradigms: Andragogy (Adult Learning Theory)<\/a>, Social Cognitive Theory<\/a>, and the Behavioral Consultation and Coaching Model. Together, these frameworks establish the foundational premise that adult skill acquisition requires collaborative, non-hierarchical, practice-based instructional architectures.<\/p>\n The foundational paradigm is Malcolm Knowles' Adult Learning Theory (Andragogy)<\/strong> (Knowles, 1984<\/a>). Knowles posited that adult learners are fundamentally distinct from pedagogical subjects: they possess self-directed self-concepts, accumulate rich experiential reservoirs that serve as learning resources, exhibit learning readiness tightly coupled to social roles, and require immediate, problem-centered applicability of knowledge. The CFS directly reflects these principles. By organizing coaching into collaborative pre-observation planning, open-ended reflective questioning, and reciprocal problem-solving, the scale penalizes top-down, authoritarian lecturing. It operationalizes a coaching dynamic where the professional serves as a reflective facilitator who honors the caregiver's expertise regarding their child while integrating evidence-based behavioral strategies into established family routines.<\/p>\n Concurrently, the CFS draws extensively from Albert Bandura's Social Cognitive Theory<\/strong> (Bandura, 1986<\/a>), particularly the mechanisms of vicarious modeling, enactive mastery experiences, and the development of perceived self-efficacy<\/em>. Self-efficacy\u2014the personal belief in one's capability to execute behaviors necessary to produce specific performance attainments\u2014is the single strongest predictor of sustained parental intervention implementation. Within the CFS architecture, enactive mastery is deliberately cultivated during the observation phase, where caregivers actively execute intervention strategies in real time. The subsequent post-observation feedback and collaborative problem-solving phases provide psychological safety, positive verbal persuasion, and attributional framing, transforming behavioral practice into robust, enduring parental self-efficacy.<\/p>\n Finally, the CFS operationalizes the empirical Triadic Behavioral Consultation and Coaching Model<\/strong> formulated within applied behavior analysis and early childhood intervention literature (e.g., Rush & Shelden, 2011; Snyder et al., 2015). This framework conceptualizes the coach-coachee-child relationship as a systemic triad wherein the ultimate beneficiary (the child with developmental disabilities) is reached through the capacity-building of the primary agent (the coachee). The triadic model insists that coaching cannot be haphazard or conversational; it requires a structured, empirical learning cycle. The CFS provides the operational blueprint for this cycle, establishing clear empirical criteria for each phase to ensure that coaching practices retain their programmatic integrity across interventions.<\/p>\n The construct validity, test sensitivity, and criterion-related integrity of the Coaching Fidelity Scale have been rigorously evaluated across empirical intervention trials (Meadan et al., 2023<\/a>), drawing observational data from three distinct multi-baseline and group-design studies focusing on naturalistic caregiver coaching (Chung et al., 2020<\/a>, 2021; Meadan et al., 2020<\/a>).<\/p>\n To ascertain whether the CFS is sensitive to meaningful differences in coaching competence, researchers conducted known-groups comparisons contrasting untrained baseline observations against post-training implementation sessions. Independent sample comparisons were performed between a baseline cohort of pre-intervention observations ($n = 50$<\/span>) and a post-intervention cohort ($n = 81$<\/span>). Given the non-normal distributional properties typical of fidelity rating data, a non-parametric Mann\u2013Whitney U<\/em> test<\/a> was executed.<\/p>\n The statistical analyses yielded a profound, statistically significant differentiation between the two conditions. The pre-intervention baseline observations exhibited minimal coaching fidelity, characterized by a mean score of $1.00$ ($SD = 1.20$<\/span>). In contrast, post-intervention fidelity ratings climbed to a mean of $5.91$ ($SD = 2.52$<\/span>). The Mann\u2013Whitney test confirmed this difference was highly significant: $$W = 327.00, \\quad p < .001$$<\/span> The associated effect size, quantified via the rank-biserial correlation<\/a> ($r_{rb}$<\/span>), was calculated at $.84$<\/span>. This extraordinarily large effect size indicates that the CFS possesses exceptional test sensitivity, demonstrating that the checklist accurately captures systematic shifts in professional coaching behavior following empirical training programs and is free of floor or ceiling artifact constraints.<\/p>\n Content validity was established through an iterative developmental process involving expert panels in speech-language pathology, special education, and applied behavior analysis. Initial implementations of the original 9-item prototype revealed that practitioners frequently struggled to engage caregivers in rigorous, ongoing adaptation of strategies when initial efforts stalled. Based on qualitative feedback and error analysis across two empirical trials (Chung et al., 2020<\/a>; Meadan et al., 2020<\/a>), the author team added two granular items targeting iterative problem-solving (specifically labeled Problem-Solving items 2 and 4). This refinement expanded the tool to 11 core items, ensuring comprehensive operational coverage of the collaborative problem-solving construct and establishing robust face and content validity.<\/p>\n The reliability of the Coaching Fidelity Scale has been extensively documented through evaluations of both internal consistency and inter-observer agreement across live and video-recorded observations.<\/p>\n The scale's internal consistency was evaluated by computing Cronbach's alpha<\/a> across the full item matrix. The CFS demonstrated outstanding internal consistency: $$\\alpha = .91 \\quad (95% \\text{ CI } [.89, .93])$$<\/span> This coefficient exceeds the standard threshold of $.80$<\/span> required for basic research instruments and satisfies the stringent $.90$<\/span> threshold demanded for high-stakes individual diagnostic and professional supervisory assessments. Furthermore, the average inter-item correlation was computed at: $$r = .47 \\quad (95% \\text{ CI } [.41, .53])$$<\/span> In psychometric scale development, an average inter-item correlation between $.40$<\/span> and $.50$<\/span> represents the optimal psychometric balance; it confirms that the items are sufficiently homogeneous to measure a coherent underlying construct without suffering from item redundancy or overly narrow operationalization.<\/p>\n Because the CFS is an observational measurement system completed by third-party evaluators, demonstrating high interrater reliability is essential to rule out observer drift and subjective scoring bias. Paired independent raters were trained to criterion using video benchmarks prior to scoring study sessions. The degree of agreement between independent raters for the overall CFS score was evaluated using the Intraclass Correlation Coefficient (ICC)<\/a>:<\/p>\n An ICC value surpassing $.96$<\/span> indicates nearly perfect agreement between independent raters, confirming that the operational definitions within the CFS checklist provide sufficiently low-inference behavioral anchors for observers to evaluate coaching fidelity with negligible measurement error.<\/p>\n To investigate the latent dimensionality of the Coaching Fidelity Scale and verify whether its component behaviors reflect a single global fidelity dimension or distinct subcomponents, an exploratory factor analysis (EFA)<\/a> was conducted on the core item matrix (Meadan et al., 2023<\/a>).<\/p>\n The exploratory factor analysis extracted a clear, uncompromised unidimensional solution<\/strong>. A single general factor accounted for 77.84%<\/strong> of the total variance across the scale items. In psychometric research, a single dominant factor accounting for over 50% (and particularly over 70%) of total variance provides decisive empirical justification for treating a scale as strictly unidimensional, confirming that the three sequential phases (pre-observation, observation, post-observation) function cohesively to reflect a solitary latent construct: overall coaching fidelity.<\/p>\n All evaluated items demonstrated high factor saturations on the single extracted factor. Every item exhibited a primary factor loading exceeding .76<\/strong> (with specific item loadings reaching up into the high .80s, exemplified by benchmark item JP1). Commensurately, item communalities ($h^2$<\/span>) were uniformly high, with all values exceeding .609<\/strong>. These elevated communalities indicate that the latent coaching fidelity factor explains more than 60% of the variance in each individual item, leaving minimal residual uniqueness or measurement error. Crucially, the analysis revealed no significant cross-loadings or anomalous secondary factor bifurcations.<\/p>\n The structural fit of the unidimensional measurement model was comprehensively assessed against conventional fit indices, with the empirical data demonstrating good-to-excellent alignment with the theoretical model across all evaluated parameters:<\/p>\n These empirical findings confirm that calculating a single, composite total fidelity score by summing the adhered items is psychometrically justified and theoretically robust.<\/p>\n The Coaching Fidelity Scale (CFS) is structured as a low-inference, criterion-referenced observational checklist. The key administrative and structural specifications of the instrument include:<\/p>\n The Coaching Fidelity Scale was formally published in 2023<\/strong> in the peer-reviewed journal Infants & Young Children<\/em> (Meadan et al., 2023<\/a>).<\/p>\n The following peer-reviewed publications provide the theoretical foundation, methodological development, and empirical psychometric validation of the Coaching Fidelity Scale:<\/p>\n The official, complete observational checklist items of the Coaching Fidelity Scale (CFS)<\/strong> are proprietary and protected under academic copyright by Wolters Kluwer Health and the study authors (Meadan et al., 2023<\/a>). The verbatim operational rating forms, detailed scoring rubric definitions, and observer training manuals are not reproduced in the open public domain.<\/p>\n2. Keywords<\/h2>\n
3. Authors<\/h2>\n
\n
4. Purpose<\/h2>\n
5. Psychological Construct<\/h2>\n
Pre-Observation Phase<\/h3>\n
\n
Observation Phase<\/h3>\n
Post-Observation Phase<\/h3>\n
\n
Optional Video Feedback Phase<\/h3>\n
6. Theoretical Framework<\/h2>\n
7. Validity<\/h2>\n
Test Sensitivity and Known-Groups Validity<\/h3>\n
Content and Social Validity<\/h3>\n
8. Reliability<\/h2>\n
Internal Consistency<\/h3>\n
Interrater Reliability (IRR)<\/h3>\n
9. Factor Analysis<\/h2>\n
Factor Extraction and Variance Explained<\/h3>\n
Factor Loadings and Communalities<\/h3>\n
Structural Fit Indices<\/h3>\n
\n
10. Instrument \/ Measurement Tool<\/h2>\n
\n
\n
\n
11. Permissions & Fee and Test Year<\/h2>\n
\n
12. References<\/h2>\n
\n
13. Items of the Scale<\/h2>\n