The Nuclear Team Skills Taxonomy<\/strong> is an evidence-based behavioral marker framework developed to identify, assess, and train non-technical and teamworking skills among control room crews and operational personnel in nuclear power plants<\/a> (NPPs). Developed by industrial psychologists Paul O'Connor, Angela O'Dea, and Rhona Flin in 2008, the taxonomy addresses the socio-technical complexities inherent in high-reliability industrial operations, where cognitive and collaborative failures represent major vulnerabilities to process safety. The instrument was derived through domain familiarization (systematic documentation reviews, control room observations, and simulator audits) followed by in-depth Critical Incident Technique<\/a> (CIT) interviews conducted with 38 licensed operations personnel across three commercial nuclear power stations in the United Kingdom. Qualitative thematic analysis of the resulting 314 behavioral statements generated a hierarchical architecture comprising five overarching categories: Shared Situation Awareness<\/em>, Team-Focused Decision Making<\/em>, Communication<\/em>, Co-ordination<\/em>, and Collaboration<\/em> (initially termed Influence), which are further operationalized into 16 discrete behavioral elements. Unlike self-report psychometric inventories, the taxonomy serves as a standardized behavioral observation system and diagnostic tool utilized primarily in full-scope simulator training, licensed operator requalification, root-cause event investigations, and organizational human factors programs. Empirical evaluations during its development demonstrated acceptable levels of inter-rater reliability among human factors coders, while establishing robust content and ecological validity. This article presents a comprehensive psychometric and operational review of the taxonomy, examining its theoretical underpinnings in high-reliability theory and distributed cognition, its psychometric properties, its scoring paradigms, and its ongoing application within international nuclear safety governance.<\/p>\n Nuclear Team Skills Taxonomy, Non-Technical Skills (NTS), Crew Resource Management, Nuclear Power Plant Operations, Situation Awareness, Naturalistic Decision Making, Team Performance Measurement, Behavioral Marker System, Human Factors, High Reliability Organizations, Critical Incident Technique, Control Room Simulator Training.<\/p>\n The Nuclear Team Skills Taxonomy was formulated through a collaborative industrial human factors research initiative conducted by researchers affiliated with the Industrial Psychology Research Centre at the University of Aberdeen, Scotland, United Kingdom:<\/p>\n The operational safety of civil nuclear energy installations hinges on the resilience of socio-technical systems wherein human operators interact with automated reactor control architectures. Historical retrospective analyses of major process catastrophes\u2014including the 1979 Three Mile Island accident<\/a> and the 1986 Chernobyl disaster<\/a>\u2014consistently identified that catastrophic escalations were rarely driven solely by isolated mechanical malfunctions or individual technical ignorance. Instead, breakdown sequences were accelerated by systemic failures in inter-operator communication, flawed mental models, procedural bottlenecks, coordination lapses, and inadequate distributed decision-making under severe operational time pressure.<\/p>\n Despite early regulatory interventions emphasizing technical proficiency and strict procedure compliance, an empirical void existed regarding the precise, observable teamwork competencies required of nuclear central control room (CCR) teams. The primary purpose of the Nuclear Team Skills Taxonomy is to provide a structured, empirically validated, and standardized framework for specifying, training, and assessing the non-technical skills of nuclear power plant operations personnel. Specifically, the taxonomy was designed to accomplish four strategic objectives:<\/p>\n The Nuclear Team Skills Taxonomy operationalizes the overarching psychological construct of Non-Technical Skills<\/em> (NTS), defined as the cognitive, social, and personal resource skills that complement technical proficiency to ensure safe and efficient task execution within high-reliability environments. In the taxonomy, team skills are structured into five core functional categories containing 16 underlying elements:<\/p>\n Situation awareness<\/a> (SA) represents the continuous perception of environmental elements, comprehension of their meaning, and projection of their status in the near future. Within nuclear control rooms, this construct is elevated to team-level or shared situation awareness. It comprises four elements:<\/p>\n This category encapsulates how control room teams identify problems, evaluate options, and commit to courses of action within both high-velocity emergencies and ambiguous, slowly developing transients. It bridges analytical reasoning with recognition-primed mechanisms through four elements:<\/p>\n Communication represents the operational transmission of semantic, technical, and intentional data across the team interface. It comprises two elements:<\/p>\n Co-ordination pertains to the structural and temporal synchronization of human actions across plant systems, ensuring seamless workflow integration across three elements:<\/p>\n Originally designated as “Influence,” this category reflects the relational dynamics, leadership architecture, and mutual interdependence necessary to maintain operational cohesion across three elements:<\/p>\n The Nuclear Team Skills Taxonomy is grounded in the convergence of four major organizational and human factors paradigms: High Reliability Organizations (HRO) theory, Distributed Team Cognition, Naturalistic Decision Making (NDM), and the Non-Technical Skills (NTS) behavioral marker tradition.<\/p>\n Originating from the seminal work of Weick, Sutcliffe, and colleagues, HRO theory posits that organizations operating in inherently unforgiving, high-hazard environments\u2014such as nuclear power plants, aircraft carriers, and air traffic control systems\u2014succeed in avoiding catastrophe by cultivating collective mindfulness. This orientation is defined by five central capabilities: preoccupation with failure, reluctance to simplify interpretations, sensitivity to operations, commitment to resilience, and deference to expertise. The Nuclear Team Skills Taxonomy translates these philosophical tenets into observable operational behaviors. For example, the elements of Anticipation<\/em> and Maintain Overview<\/em> reflect sensitivity to operations and preoccupation with failure, whereas Assertiveness<\/em> and Deference to Expertise<\/em> permit junior technicians to interrupt senior authority if an operational barrier is compromised.<\/p>\n The framework draws heavily from distributed cognition<\/a> and the shared mental model<\/a> literature (Cannon-Bowers, Salas, & Converse, 1993; Endsley, 1995). In a nuclear power plant, cognitive processing is not confined to the mind of a single engineer; it is distributed across multiple agents (control engineers, shift managers, field technicians) and artifacts (analog indicator boards, computerized safety displays, paper-based EOPs). Effective team functioning demands overlapping mental representations regarding system architecture, task dependencies, and teammate capabilities. The taxonomy's emphasis on Develop Understanding<\/em>, Information Exchange<\/em>, and Supporting Behaviour<\/em> provides the behavioral mechanisms through which disparate individuals maintain congruency among their respective internal models under changing environmental demands.<\/p>\n Classical normative decision models that emphasize exhaustive, linear option comparison break down in complex, time-pressured industrial environments. The taxonomy incorporates Gary Klein's Recognition-Primed Decision<\/a> (RPD) framework within its Team Focused Decision Making<\/em> category. In nuclear operations, highly experienced operators rely on holistic pattern recognition to rapidly categorize operational anomalies into known typologies, immediately executing associated recovery scripts (Intuitive Decision Making<\/em>). Conversely, during novel, unmodeled, or cross-system plant disruptions, crews shift to deliberate causal reasoning, formal probabilistic evaluations, and team hypothesis testing (Analytical Decision Making<\/em>). The taxonomy recognizes both modes of cognitive reasoning as complementary.<\/p>\n Directly following the methodological paradigm established by Rhona Flin and the European civil aviation consortium in the development of NOTECHS (Non-Technical Skills for Pilots), the Nuclear Team Skills Taxonomy was constructed to avoid abstract personality inventories. It is rooted in behavioral marker system methodology, which relies on concrete, observable, and trainable behavioral indicators. By organizing complex psychosocial interactions into hierarchical categories and discrete operational elements, the taxonomy grounds psychological assessment in observable behavioral outcomes directly actionable in simulation environments.<\/p>\n The psychometric integrity of behavioral observation systems requires specialized validation paradigms distinct from self-report instruments. The Nuclear Team Skills Taxonomy exhibits strong evidence across multiple validity domains:<\/p>\n Content validity was established through the two-phase empirical derivation methodology executed by O'Connor et al. (2008). In Phase 1, intensive domain familiarization was conducted across operating nuclear facilities, including structural plant walk-throughs, comprehensive audits of utility operating documentation and training syllabi, simulator exercise observations, and structured role interviews with operational personnel. This ensured that the conceptual taxonomy captured genuine operational realities rather than theoretical abstractions.<\/p>\n In Phase 2, the researchers utilized the Critical Incident Technique<\/a> (CIT)\u2014a method initially formulated by John Flanagan (1954) for identifying critical job behaviors. Qualitative interviews were conducted with 38 operational personnel across three British nuclear power plants (representing diverse reactor technologies, including Advanced Gas-cooled Reactors [AGRs] and Pressurized Water Reactors [PWRs]). The sample incorporated Shift Charge Engineers (n=11), Control Room Supervisors (n=10), Desk Engineers (n=11), and field-level Operations Technicians (n=6). Interviewees described non-routine events, emergent reactor transients, and complex plant realignments. From these interviews, 314 concrete behavioral statements regarding team skills were extracted. A cross-functional panel of human factors experts and seasoned nuclear operations personnel iteratively sorted, classified, and synthesized these statements, ensuring strong ecological validity and alignment with control room operational environments<\/span>.<\/p>\n Construct validity is evidenced through the alignment of the 16 elements with validated theoretical dimensions established in the broader industrial teamwork literature, specifically Salas et al.'s (2005) “Big Five” model of teamwork (team leadership, mutual performance monitoring, backup behavior, adaptability, and team orientation) and Endsley's (1995) multi-level situation awareness framework.<\/p>\n Subsequent criterion-related validation studies across international nuclear training facilities have demonstrated that crews exhibiting superior non-technical skill ratings\u2014specifically within Building Situation Awareness<\/em> and Co-ordination<\/em>\u2014exhibit significantly faster identification of simulated secondary circuit breaches, fewer procedural violations during emergency depressurizations, and reduced time-to-stabilize during reactor trips. The taxonomy discriminates effectively between novice (e.g., student or cross-training engineers) and expert control room crews, supporting its discriminative construct validity.<\/p>\n Because the Nuclear Team Skills Taxonomy is designed as an observational behavioral categorization and assessment rubric rather than a self-administered psychometric test, reliability centers on inter-rater reliability<\/a> and classification consistency across human assessors.<\/p>\n During the initial derivation and coding phase, O'Connor et al. (2008) tested the categorization reliability of the 314 CIT behavioral statements. Independent human factors researchers were provided with the category and element definitions and tasked with blindly assigning a random sample of the extracted statements to the taxonomic structure. Inter-rater reliability was assessed using Cohen's kappa<\/a> (\u03ba) to correct for chance agreement. The coding process yielded acceptable levels of inter-rater agreement across the taxonomic categories, with kappa values exceeding the conventional 0.70 benchmark (ranging from \u03ba = 0.72 to \u03ba = 0.84 depending on the specific element tier), confirming that the taxonomic boundaries were distinct and conceptually coherent<\/span>.<\/p>\n As noted in the original research, while acceptable coding reliability was achieved during taxonomy construction, applying the taxonomy for live simulator assessment requires calibrated training. Inter-assessor reliability in operational settings is vulnerable to human cognitive biases, including the halo effect<\/a> (a general favorable impression distorting specific behavioral dimensions), central tendency bias, and domain-expertise discrepancies between licensed technical instructors and human factors specialists.<\/p>\n To establish high inter-rater reliability (Intraclass Correlation Coefficients [ICC] > 0.80) in practice, utility instructors must complete standardized Frame-of-Reference (FOR) training. FOR training calibrates evaluators using benchmark video recordings of simulated reactor transients, establishing standardized scoring thresholds for “Unsatisfactory,” “Marginal,” “Acceptable,” and “Exemplary” team performance<\/span>.<\/p>\n Unlike psychometric scales composed of continuous survey items, behavioral marker systems do not typically follow standard questionnaire-based Exploratory Factor Analysis<\/a> (EFA) or Confirmatory Factor Analysis<\/a> (CFA) during their formative stage. Instead, their latent structure is derived through inductive qualitative classification, card-sorting tasks, and thematic clustering algorithms applied to operational statements.<\/p>\n The taxonomy's structural validity rests on a hierarchical five-factor taxonomy. The 314 behavioral statements were sorted into discrete, non-redundant categories. The structural taxonomy is mapped as follows:<\/p>\n In subsequent simulation trials, exploratory and confirmatory factor analyses of behavioral marker observation rubrics across related domains (such as NOTECHS in aviation and ANTS in healthcare) have revealed high inter-factor correlations (often r<\/em> > .60) among cognitive dimensions (Situation Awareness and Decision Making) and social dimensions (Communication and Co-ordination). This reflects the reality that effective non-technical performance operates as an integrated behavioral network: clear communication is a prerequisite for shared situation awareness, which directly enables collaborative decision-making and dynamic co-ordination<\/span>.<\/p>\n The Nuclear Team Skills Taxonomy is operationalized in training and operational settings as a structured, criterion-referenced behavioral observation rating system. Below is the structural configuration and operational protocol of the tool:<\/p>\n The research establishing the Nuclear Team Skills Taxonomy was published in 2008<\/strong> in the International Journal of Industrial Ergonomics<\/em> (Elsevier). The conceptual taxonomy, category definitions, and illustrative behavioral elements published within the academic literature are protected under standard scientific copyright; however, the theoretical framework and operational structures are widely accessible for academic, scientific, and non-commercial research purposes.<\/p>\n Commercial deployment, proprietary training adaptation, or utility-wide integration of customized behavioral marker software within nuclear plant training programs must adhere to intellectual property conditions governed by the publisher and the original authors. Nuclear utilities, international safety agencies (such as the International Atomic Energy Agency<\/a> [IAEA] and the Institute of Nuclear Power Operations [INPO]), and research institutions frequently adapt the public taxonomic principles to suit regional regulatory frameworks and plant configurations (e.g., AGR, PWR, BWR, CANDU, VVER).<\/p>\n The Nuclear Team Skills Taxonomy is an observational behavioral marker system structured across 5 overarching categories and 16 behavioral elements. During simulation-based assessments, evaluators utilize the following operational elements, definitions, and associated behavioral marker indicators to evaluate control room crew performance:<\/p>\n The collective cognitive process of perceiving, comprehending, and projecting plant states and task environments.<\/em><\/p>\n Definition:<\/strong> Analyzing and sharing information in order to develop an accurate model of the problem or operational task.<\/p>\n Definition:<\/strong> Forward planning to identify and discuss contingency strategies and\/or possible future plant problems.<\/p>\n Definition:<\/strong> Retaining a broad picture of a task or situation without becoming excessively involved in low-level details.<\/p>\n Definition:<\/strong> Observing and checking the activities, execution, and physiological\/cognitive performance of other team members.<\/p>\n The process of selecting a strategy and committing to action based on operational evidence, procedural guidelines, and shared consensus.<\/em><\/p>\n Definition:<\/strong> Gathering and integrating information from team members, selecting the optimal solution, and systematically evaluating consequences.<\/p>\n Definition:<\/strong> Adhering to written Emergency, Abnormal, and Standard Operating Procedures with appropriate verification.<\/p>\n Definition:<\/strong> Associating environmental cues and alarm configurations to appropriate corrective actions and initiating rapid decisions.<\/p>\n Definition:<\/strong> Using professional judgment to make decisions and execute actions without waiting for explicit orders when safety dictates.<\/p>\nKeywords<\/h2>\n
Authors<\/h2>\n
\n
Purpose<\/h2>\n
\n
Psychological Construct<\/h2>\n
1. Building Situation Awareness<\/h3>\n
\n
2. Team-Focused Decision Making<\/h3>\n
\n
3. Communication<\/h3>\n
\n
4. Co-ordination<\/h3>\n
\n
5. Collaboration<\/h3>\n
\n
Theoretical Framework<\/h2>\n
High Reliability Organizations (HRO) and Collective Mindfulness<\/h3>\n
Distributed Cognition and Shared Mental Models<\/h3>\n
Naturalistic Decision Making (NDM) and the Recognition-Primed Decision (RPD) Model<\/h3>\n
The Behavioral Marker Tradition (NOTECHS Heritage)<\/h3>\n
Validity<\/h2>\n
Content and Ecological Validity<\/h3>\n
Construct and Criterion-Related Validity<\/h3>\n
Reliability<\/h2>\n
Inter-Rater Agreement in Taxonomy Development<\/h3>\n
Observational Reliability in Training Settings<\/h3>\n
Factor Analysis and Structural Evaluation<\/h2>\n
Structural Architecture<\/h3>\n
\n
Instrument \/ Measurement Tool<\/h2>\n
\n
\n
\n
Permissions, Fee, and Test Year<\/h2>\n
References<\/h2>\n
\n
Items of the Scale<\/h2>\n
Category 1: Shared Situation Awareness<\/h3>\n
1. Develop Understanding<\/h4>\n
\n
2. Anticipation<\/h4>\n
\n
3. Maintain Overview<\/h4>\n
\n
4. Performance Monitoring<\/h4>\n
\n
Category 2: Team Focused Decision Making<\/h3>\n
5. Analytical Decision Making<\/h4>\n
\n
6. Procedure Following<\/h4>\n
\n
7. Intuitive Decision Making<\/h4>\n
\n
8. Initiative<\/h4>\n
\n