The Daily Experience of Simulation Training (DEST)<\/strong>, developed by Chiu, Cross, Wheeler, Evans, and Goulet (2023)<\/a>, is an ecologically sensitive, multi-component psychometric instrument designed to capture daily fluctuations in self-efficacy, pedagogical feedback perceptions, and qualitative professional reflections among child welfare professionals undergoing immersive simulation-based education. Constructed to evaluate cohort-level learning trajectories at the Child Protection Training Academy (CPTA)—a collaborative venture established by the University of Illinois at Springfield (UIS) and the Illinois Department of Children and Family Services (DCFS)—the DEST operationalizes professional growth in high-stakes child welfare contexts. The core quantitative engine of the instrument, the Daily Experience of Simulation Training – Confidence Subscale (DEST-CS)<\/em>, comprises 13 distinct items evaluated on a 7-point Likert scale (ranging from 1 = lowest confidence to 7 = highest confidence), embedded within a broader 20-item daily tracking battery that includes multi-source trainer and standardized actor feedback appraisal alongside an open-ended narrative reflection log. Psychometric evaluations conducted across longitudinal time points during intensive, week-long investigative simulation boot camps demonstrate outstanding reliability and structural stability. Specifically, principal component analyses with varimax rotation across six distinct assessment waves yielded a persistent unifactorial confidence construct, accounting for substantial portions of total variance ranging from 71.4% (eigenvalue = 9.28) to 76.5% (eigenvalue = 9.94). Internal consistency reliability estimates were extraordinarily high, with Cronbach’s alpha coefficients exceeding α = .95 at every single administration point. The DEST bridges the gap between traditional pre-\/post-training evaluation models and dynamic, intra-individual process tracking, affording researchers and workforce developers an empirically validated framework for monitoring deliberate skill acquisition, procedural confidence, and experiential learning in child protective services<\/span>.<\/p>\n Child Welfare, Professional Competence, Simulation Training Effectiveness, Child Protection Trainees, Self-Efficacy Measurement, Longitudinal Daily Tracking, Program Evaluation, Protective Services, Deliberate Practice, Factor Analysis, Psychometrics, Experiential Learning<\/p>\n The Daily Experience of Simulation Training (DEST) was conceptualized, validated, and published by an interdisciplinary team of researchers and child protection clinical educators associated with the University of Illinois system:<\/p>\n Child protection casework is characterized by immense cognitive load, high emotional volatility, ambiguous investigative boundaries, and life-altering decision-making. Historically, public child welfare agencies have relied heavily on didactic classroom instruction to onboard new child protection investigators. However, empirical literature in public administration and social work education has repeatedly highlighted the failure of traditional didactic pedagogy to bridge the “transfer of learning” chasm—leaving newly minted investigators vulnerable to acute workplace anxiety, premature turnover, procedural error, and severe burnout. To transform this training paradigm, the Child Protection Training Academy (CPTA) integrated experiential learning and clinical simulation paradigms originally pioneered within aerospace and medical education, constructing mock physical environments (such as standardized residential home labs, forensic interview rooms, and simulated courtrooms) with professional standardized actors (Goulet et al., 2020<\/a>).<\/p>\n The primary purpose of the Daily Experience of Simulation Training (DEST) is to provide a robust, daily repeated-measures psychometric evaluation protocol specifically engineered for week-long simulation academies. Unlike static pre-test\/post-test instruments that mask non-linear learning curves, temporary dips in confidence (such as the “unconscious incompetence to conscious incompetence” transition), or daily instructional bottlenecks, the DEST measures day-to-day fluctuations in trainee confidence and learning experience. Its administrative and clinical purposes include:<\/p>\n The DEST operationalizes a multidimensional yet structurally integrated assessment of task-specific professional self-efficacy<\/em>, instructional feedback utility<\/em>, and metacognitive reflection<\/em> within an immersive occupational learning ecology. In psychometrics, professional self-efficacy does not represent a generalized, diffuse personality trait; rather, it manifests as a contextualized belief in one’s capability to mobilize cognitive, behavioral, and emotional resources required to meet specific situational demands.<\/p>\n The foundational quantitative core of the instrument (Items 1–13) measures task-specific self-efficacy across critical operational domains encountered by child protection investigators. Rather than assessing abstract legal knowledge, these items measure the perceived self-competence to execute complex, dual-purpose social-investigative tasks:<\/p>\n The secondary quantitative dimension assesses the trainee’s subjective valuation of feedback provided across distinct educational roles. Grounded in adult learning theory, the DEST captures how trainees process feedback from classroom instructors, simulation trainers, and standardized actors playing familial roles (e.g., “Mother Figure,” “Paramour Figure,” and “Other Adult Caregiver”). This dimension captures the educational impact of receiving immediate, unscripted feedback directly from the perspective of simulated clients.<\/p>\n The qualitative narrative dimension captures cognitive assimilation through a focused journaling prompt. By articulating the most meaningful concepts or skills acquired each day, trainees engage in self-regulation, transforming raw simulation stress into structured professional competence.<\/p>\n The theoretical architecture of the Daily Experience of Simulation Training is deeply rooted in three converging frameworks of cognitive psychology, educational theory, and occupational competence: Bandura’s Social Cognitive Theory<\/strong>, Kolb’s Experiential Learning Model<\/strong>, and Ericsson’s Theory of Deliberate Practice<\/strong>.<\/p>\n 1. Experiential Learning Cycle (Kolb):<\/strong> Immersion in standardized residential\/courtroom simulation → Immediate cognitive\/emotional confrontation with complex child safety scenarios.<\/p>\n 2. Deliberate Practice & Feedback (Ericsson):<\/strong> Structured debriefing → Multi-angle critique from trainers and standardized client actors → Error correction and behavioral adjustment.<\/p>\n 3. Self-Efficacy Calibration (Bandura):<\/strong> Mastery experiences + Vicarious observation → Daily recalibration of self-efficacy measured via DEST-CS → Narrative metacognitive consolidation via reflective log.<\/p>\n<\/p><\/div> Albert Bandura’s (1997)<\/a> self-efficacy construct posits that an individual’s belief in their ability to succeed in specific situations governs how they think, behave, and feel. Self-efficacy is cultivated through four primary informational sources: mastery experiences<\/em>, vicarious experiences<\/em>, verbal persuasion<\/em>, and physiological\/affective states<\/em>. Simulation training in child welfare provides direct mastery experiences through high-fidelity role-play, vicarious experiences through peer observation, verbal persuasion via targeted expert debriefing, and emotional desensitization to acute workplace arousal. The DEST measures the daily calibration of this self-efficacy construct, capturing how confidence shifts from baseline anxiety to grounded professional agency as trainees encounter realistic family resistance, domestic violence scenarios, and judicial interrogation.<\/p>\n David Kolb’s (1984)<\/a> experiential learning theory delineates a four-stage cyclical model: Concrete Experience, Reflective Observation, Abstract Conceptualization, and Active Experimentation. The structure of the DEST parallels this cycle exactly. Trainees undergo a Concrete Experience in the simulation lab, engage in Reflective Observation during debriefings, formulate Abstract Conceptualizations aided by multi-trainer feedback, and engage in Active Experimentation the following training day. The DEST functions as an explicit metacognitive instrument that formalizes the Reflective Observation and Abstract Conceptualization phases via daily psychometric evaluation and narrative logging.<\/p>\n K. Anders Ericsson’s (1993)<\/a> deliberate practice model establishes that true professional expertise is not accumulated merely through passive tenure, but through targeted, highly structured practice paired with immediate informational feedback, error detection, and repetitive refinement. Simulation training operationalizes deliberate practice in social welfare. The DEST captures the educational mechanisms of deliberate practice by holding trainees accountable to multi-perspective feedback (from trainers and actors alike), isolating concrete operational competencies, and systematically measuring self-efficacy growth day over day.<\/p>\n The psychometric validation of the Daily Experience of Simulation Training (DEST) was executed within a rigorous statewide evaluation framework involving cohorts of newly hired child protection investigators entering the Illinois Department of Children and Family Services (DCFS) through the Child Protection Training Academy (Chiu et al., 2023<\/a>).<\/p>\n Content validity was established through collaborative, expert-driven curricular design. The 13 confidence items were derived directly from statutory mandates, Illinois DCFS core practice models, forensic interviewing protocols, and extensive job analysis interviews conducted with veteran investigative supervisors, administrative judges, and clinical child welfare educators. Each item maps directly to a high-frequency, high-liability competency required during the initial 60-day child abuse\/neglect investigation window. Face validity was confirmed via cognitive walkthroughs with pilot cohorts of novice caseworkers who confirmed that the DEST items accurately reflected their daily cognitive challenges, emotional hurdles, and procedural duties.<\/p>\n Construct validity is evidenced through the tool’s exceptional structural stability and factorial invariance across multiple assessment points. In longitudinal evaluation designs, repeated-measures instruments frequently suffer from construct drift or temporal instability. As detailed in the psychometric analyses conducted by Chiu and colleagues (2023), principal component analyses across all six daily measurement waves (Day 1 baseline morning through Day 5 post-simulation) demonstrated that the 13 items consistently load on a singular, dominant self-efficacy dimension. The factor structure retained strong conceptual coherence across time, confirming that the tool reliably measures the same latent professional confidence construct across the full duration of training.<\/p>\n A crucial validity requirement for any training evaluation tool is sensitivity to experiential intervention (responsiveness). The DEST demonstrated significant, progressive upward trajectories in mean scores across training days. Novice investigators exhibited moderate-to-low baseline confidence on Day 1 morning (particularly on complex items such as testifying in court, framing protective custody, and identifying subtle caregiver deception). As simulation scenarios progressed, mean item scores systematically increased, accompanied by standard deviation compression, reflecting converging group competence and successful instructional mastery.<\/p>\n The reliability of the Daily Experience of Simulation Training – Confidence Subscale (DEST-CS) has been demonstrated through internal consistency analyses across six distinct longitudinal assessment waves. In high-stakes human service measurement, an internal consistency coefficient exceeding .80 is considered strong, while values above .90 reflect gold-standard precision suitable for individual-level diagnostic and evaluative applications.<\/p>\nKeywords<\/h2>\n
Authors<\/h2>\n
\n
chiu22@illinois.edu<\/code>).<\/li>\nPurpose<\/h2>\n
\n
Psychological Construct<\/h2>\n
1. Investigative Self-Efficacy and Procedural Confidence (DEST-CS)<\/h3>\n
\n
2. Multisource Pedagogical Feedback Receptivity<\/h3>\n
3. Reflective Metacognitive Log<\/h3>\n
Theoretical Framework<\/h2>\n
Conceptual Framework: Integrated Architecture of the DEST<\/h4>\n
1. Bandura’s Self-Efficacy Theory<\/h3>\n
2. Kolb’s Experiential Learning Cycle<\/h3>\n
3. Ericsson’s Theory of Deliberate Practice<\/h3>\n
Validity<\/h2>\n
1. Content and Face Validity<\/h3>\n
2. Construct and Structural Validity<\/h3>\n
3. Longitudinal Sensitivity to Change (Criterion Validity)<\/h3>\n
Reliability<\/h2>\n