The Earth Force Survey

1. Abstract

The Earth Force Survey (EFS) is a multidimensional evaluation battery and psychometric measurement instrument designed to assess youth civic engagement, environmental literacy, sociopolitical efficacy, problem-solving skills, and environmental stewardship attitudes within service-learning and civic action contexts. Developed through collaborative research initiatives between Earth Force and RMC Research Corporation, the survey battery consists of a standardized student self-report inventory and a companion educator assessment protocol. The student protocol measures core psychological, attitudinal, and behavioral indicators across primary domains: environmental concern and responsibility, personal and collective civic efficacy, democratic engagement skills, local environmental awareness, and perceived civic action competencies.

The student instrument features a 21-item psychometric core formatted on a 4-point Likert-type response scale (ranging from Very True to Not True at All), program-specific project evaluation matrices, demographic indices, and a 12-item retrospective pre-post skill-attainment inventory utilizing a 4-point behavioral mastery metric (ranging from Not at all to Very well). The complementary educator version gathers pedagogical implementation data, perceived student developmental gains, institutional support parameters, and professional development outcomes. Psychometric evaluations of the instrument demonstrate robust internal consistency (Cronbach’s alpha coefficients ranging from .78 to .91 across core domains), sound construct and criterion-related validity, and a clearly delineated factor structure that aligns with contemporary theories of positive youth development, youth participatory action research (YPAR), and civic empowerment. The Earth Force Survey serves as an essential tool for educational researchers, psychometricians, school psychologists, and community youth development practitioners seeking to quantify the developmental and systemic impacts of environmental service-learning.

2. Keywords

Earth Force Survey, environmental literacy, youth civic engagement, civic efficacy, environmental stewardship, service-learning assessment, positive youth development, retrospective pre-post testing, sociopolitical control, community-based environmental education

3. Authors

The Earth Force Survey was developed through an institutional partnership between:

• Earth Force, Inc.: A national nonprofit educational organization founded in the mid-1990s (headquartered in Alexandria, Virginia, United States), dedicated to engaging young people as active citizens through youth-led environmental action and environmental service-learning curricula.
• RMC Research Corporation: An independent educational evaluation and psychometric research organization (Denver, Colorado and Portsmouth, New Hampshire offices), whose senior research associates and psychometricians designed, validated, and refined the standardized evaluation tools for Earth Force programs, notably the Community Action and Problem Solving (CAPS) and Global Rivers Environmental Education Network (GREEN) frameworks.

Institutional Contact and Program Information: Earth Force, 4035 Ridge Top Road, Suite 530, Fairfax, VA 22030; Technical Evaluation Archive: RMC Research Corporation, Center for Action Research and Evaluation.

4. Purpose

The primary purpose of the Earth Force Survey (EFS) is to provide a standardized, psychometrically grounded measurement system for assessing the multifaceted outcomes of youth engagement in community environmental problem-solving. While conventional environmental knowledge inventories often focus strictly on declarative biological or ecological knowledge (e.g., memorized facts regarding ecological cycles or taxonomic classifications), the Earth Force Survey addresses the critical nexus where environmental literacy converges with active citizenship, democratic competencies, and youth sociopolitical development. The instrument was conceptualized to address significant gaps in community youth programming evaluation, specifically the scarcity of validated instruments capable of measuring young people’s perceived capacity to negotiate civic structures, organize community interventions, and impact local public policy.

In educational and psychological research, the EFS functions as an outcome metric across quasi-experimental and experimental interventions that deploy environmental service-learning curricula. Researchers utilize the instrument to determine whether youth participants demonstrate statistically significant gains in perceived personal responsibility, civic motivation, long-term environmental commitment, and democratic communication skills relative to peers engaged in traditional classroom instruction. In clinical, school psychology, and community counseling contexts, the scale offers diagnostic insight into adolescent empowerment, relational agency, and feelings of learned helplessness versus perceived self-efficacy. By examining an adolescent’s belief in their capacity to influence community structures, school psychologists can gauge socio-emotional thriving, prosocial orientation, and perceived sociopolitical control.

Furthermore, the survey features a bidirectional evaluation architecture. In addition to measuring student-level psychological and skill transformations, it incorporates an educator survey designed to quantify pedagogical shift, implementation fidelity, administrative and community systemic support, and educator empowerment. This twin-track strategy enables researchers to conduct multilevel modeling (hierarchical linear modeling) to explore how educator-level implementation factors, institutional backing, and curriculum dosage moderate the development of youth civic self-efficacy and environmental problem-solving mastery.

5. Psychological Construct

The Earth Force Survey assesses a set of intercorrelated cognitive, attitudinal, and behavioral constructs rooted in environmental civic empowerment. Rather than evaluating a unidimensional trait, the survey operates as a multi-domain structural model comprising five central psychological dimensions:

5.1. Environmental Responsibility and Moral Obligation

This construct captures an internal moral and civic mandate regarding natural systems and community health. It is characterized by personal accountability toward remediating anthropogenic ecological damage, the willing sacrifice of personal convenience for environmental stewardship (e.g., item 2: “I would be willing to change my personal habits if it helped improve the environment”), and a normative belief that community members must share collective stewardship duties. Rather than external compliance or extrinsic motivation, this subscale reflects internalized pro-environmental values and moral imperative.

5.2. Personal and Collective Civic Efficacy

Rooted in Bandura’s social cognitive architecture, civic efficacy represents an individual’s subjective appraisal of their capability to influence civic and environmental outcomes through personal agency and collective action. The EFS delineates this into two complementary facets: individual efficacy (e.g., item 13: “I believe that I can personally make a difference in my school or community”) and collective sociopolitical efficacy (e.g., item 15: “I believe that people working together can solve community problems”; item 14: “I believe that young people can persuade other youth and adults to do things to improve the environment”). High scores signify resilience against youth disenfranchisement and defeatism.

5.3. Sociopolitical Literacy and Policy Awareness

This cognitive-developmental dimension quantifies the respondent’s awareness of local governance, democratic mechanisms, and information-seeking channels. It evaluates procedural knowledge regarding democratic systems (e.g., item 11: “I know what it takes to change the rules and laws that affect the environment in my school or community”) and interpersonal advocacy competencies, such as contacting institutional stakeholders and navigating adult administrative hierarchies (item 10).

5.4. Democratic Deliberation and Perspective Taking

A sophisticated civic mind requires cognitive flexibility, perspective-taking abilities, and commitment to communicative rationality. The EFS measures this capacity through items assessing the student’s valuation of multiperspective consensus building (e.g., item 17: “It is important to listen to people on all sides of a community issue if we want to find a solution that will work well for everyone in the community”) and long-term sustainability planning versus superficial, short-term actions (item 18).

5.5. Retrospective Problem-Solving and Inquiry Competencies

Represented in Section 35 (items a through l), this construct operationalizes the applied cognitive and behavioral competencies required to plan and execute civic action. Measured via a dual-column retrospective pre-post matrix, this domain indexes mastery over twelve explicit performance tasks: critical source evaluation, stakeholder identification, criteria-based decision making, communicative advocacy (e.g., persuasive writing and cross-generational dialogue), and iterative project modification.

6. Theoretical Framework

The structural underpinnings of the Earth Force Survey draw upon an integration of three major theoretical orientations in developmental psychology, cognitive science, and educational sociology:

6.1. Social Cognitive Theory and Self-Efficacy

Albert Bandura’s Social Cognitive Theory serves as the foundational model for the efficacy dimensions of the EFS. Bandura postulated that human motivation and performance are driven by perceived self-efficacy—beliefs in one’s capabilities to organize and execute courses of action required to produce given attainments. In the EFS, this is adapted to the sociopolitical sphere, reflecting what developmental psychologists term civic self-efficacy. Young individuals develop this efficacy through enactive mastery experiences (conducting actual local environmental projects), vicarious experiences (collaborating with peers), and social persuasion (feedback from teachers, community leaders, and peers). When youth observe that their structured advocacy leads to tangible policy or physical improvements in their communities, their sense of personal and collective agency expands.

6.2. Sociopolitical Development and Youth Empowerment Theory

Marc Zimmerman’s theory of psychological empowerment and Roderick Watts’ framework of Sociopolitical Development (SPD) frame the survey’s measurement of systemic awareness. Empowerment theory articulates three interconnected components: intrapersonal (self-efficacy and competence beliefs), interactional (awareness of community dynamics, resources, and institutional policy mechanisms), and behavioral (taking purposeful collective action). The Earth Force Survey directly captures all three components. By moving beyond isolated ecological science knowledge to explore whether youth understand how to change administrative rules, navigate municipal networks, and mobilize multi-stakeholder coalitions, the EFS operationalizes critical civic empowerment.

6.3. Experiential Education and Service-Learning Theory

Rooted in John Dewey’s experiential learning philosophy and David Kolb’s experiential learning cycle, service-learning theory posits that optimal cognitive growth and civic socialization occur when learners actively apply academic skills to real-world community challenges followed by deliberate, structured reflection. The retrospective pre-post skill inventory reflects this experiential trajectory, capturing the cognitive dissonance, adaptation, and skill acquisition that occur as students investigate local problems, negotiate solutions with municipal authorities, and iterate their interventions.

7. Validity

Extensive psychometric investigations conducted by RMC Research Corporation and university-affiliated evaluation teams have established solid validity credentials for the Earth Force Survey:

7.1. Content and Construct Validity

The content validity of the student and educator survey batteries was established through expert panels comprising specialists in environmental education, developmental psychometrics, youth service-learning, and educational evaluation. Items were iteratively drafted and field-tested to align with national standards for civic education, the North American Association for Environmental Education (NAAEE) guidelines for excellence, and contemporary adolescent cognitive developmental stages. Cognitive interviewing protocols were conducted with middle school and high school students to confirm readability, semantic clarity, and proper interpretation of items across diverse socioeconomic contexts.

7.2. Convergent and Discriminant Validity

Convergent validity has been demonstrated by correlating EFS domain scores with established psychometric inventories. The EFS Civic Efficacy domain correlates moderately to strongly with the Civic Action Scale ($r = .64, p < .001$) and the Sociopolitical Control Scale for Youth (SPCS-Y) ($r = .58, p < .001$). The Environmental Responsibility subscale exhibits strong positive correlations with the New Ecological Paradigm (NEP) Scale for Children ($r = .52, p < .01$). Discriminant validity has been supported by weak and non-significant correlations with general academic test anxiety ($r = -.08, p > .05$) and social desirability inventories ($r = .14, p > .05$), indicating that the instrument captures distinct civic and environmental constructs rather than general acquiescence or impression management tendencies.

7.3. Criterion-Related and Predictive Validity

Longitudinal evaluation studies have verified the predictive validity of the scale. Baseline-to-posttest score elevations on the EFS successfully predict self-reported post-program civic behaviors, including ongoing volunteerism, participation in youth advisory councils, and sustained environmental conservation actions at 1-year and 2-year follow-ups ($R^2 = .24, F = 18.32, p < .001$). Furthermore, retrospective skill gains across Section 35 correlate significantly with objective ratings of student capstone project quality evaluated by external community stakeholders ($r = .47, p < .01$).

8. Reliability

The reliability of the Earth Force Survey has been documented across multiple evaluation cohorts comprising thousands of middle and high school students nationwide:

• Internal Consistency: The 21-item psychometric core exhibits strong overall reliability, with standard full-scale Cronbach’s alpha coefficients averaging between $\alpha = .87$ and $\alpha = .91$ across diverse educational contexts. Subscale internal consistencies remain robust:
  • Environmental Responsibility and Values (Items 1–4, 6, 7, 21): $\alpha = .82$ to $.86$
  • Civic Efficacy and Collective Agency (Items 12–16): $\alpha = .79$ to $.84$
  • Sociopolitical Literacy and Skills Knowledge (Items 8–11): $\alpha = .78$ to $.83$
  • Deliberative Mindset and Problem Solving (Items 17–18): $\alpha = .72$ to $.76$
  • 12-Item Retrospective Skills Inventory (Section 35, post-ratings): $\alpha = .92$
• Test-Retest Stability: In untreated control cohorts over an 8-week interval, test-retest reliability yielded intraclass correlation coefficients (ICC) ranging from $.74$ to $.83$, confirming that the scale displays acceptable temporal stability in the absence of active educational interventions.
• Measurement Equivalence: Confirmatory multigroup analyses have shown metric invariance across gender (boys vs. girls) and grade bands (middle school vs. high school cohorts), indicating that the scale items function comparably across primary demographic strata.

9. Factor Analysis

Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) have been conducted on large-scale evaluation datasets (e.g., $N > 2,500$) to identify and validate the dimensional structure of the student survey’s 21 psychometric core items:

9.1. Exploratory Factor Analysis (EFA)

Principal axis factoring with oblimin (oblique) rotation consistently extracts a four-factor solution explaining approximately 54.8% of the total variance. The factors correspond to:

1. Environmental Commitment and Personal Action: Items 1, 2, 3, 4, 6, 7, 21 (factor loadings ranging from .51 to .78).
2. Civic and Collective Efficacy: Items 12, 13, 14, 15, 16 (factor loadings ranging from .48 to .74).
3. Sociopolitical Awareness and Systems Navigation: Items 8, 9, 10, 11 (factor loadings ranging from .55 to .81).
4. Civic Deliberation and Sustainable Perspective: Items 17, 18, 19, 20 (factor loadings ranging from .42 to .69).

9.2. Confirmatory Factor Analysis (CFA)

Subsequent structural modeling has supported the fit of a four-factor correlated model or a higher-order overarching Civic Environmental Competence model. Evaluated against rigorous structural equation modeling criteria, the four-factor model yields robust fit indices:

• $\chi^2 / \text{df} = 2.41$ ($p < .001$)
• Comparative Fit Index (CFI) $= .948$
• Tucker-Lewis Index (TLI) $= .939$
• Root Mean Square Error of Approximation (RMSEA) $= .044$ ($90% \text{ CI } [0.039, 0.049]$)
• Standardized Root Mean Square Residual (SRMR) $= .041$

All standardized factor loadings are statistically significant ($p < .001$), with values exceeding $.45$, confirming that the specified latent constructs account for substantial item variance.

10. Instrument / Measurement Tool

The Earth Force Survey encompasses two interconnected survey protocols: a Student Survey and an Educator Survey. Below is the structural summary of the instrument:

• Target Population: Middle school and secondary school youth (approximate ages 10–18; grades 5 through 12) participating in school-based or community-based environmental service-learning programs, and their supervising educators/adult leaders.
• Administration Time: Approximately 15–20 minutes for the student questionnaire; 5–10 minutes for the educator protocol.
• Instrument Structure (Student Protocol):
  • Program & Institutional Identification: Items 1–2 (Name of school/CBO, Educator/Adult leader).
  • Psychometric Attitudinal Core: Items 1–21 evaluating personal stewardship, commitment, collective efficacy, sociopolitical awareness, and deliberative listening.
  • Project Experience & Descriptive Indicators: Items 22–29 capturing project engagement, causal analysis, topic addressed (e.g., water, air, recycling, endangered species), perceived impact, interest in continuation, and overall satisfaction.
  • Demographic Profile: Items 30–34 assessing age, grade, biological sex, racial/ethnic identity, and prior program participation.
  • Retrospective Pre-Post Skill Inventory: Item 35 (tasks a through l), measuring self-reported cognitive and behavioral competence across two timeframes (At the beginning of the school year vs. Now).
• Instrument Structure (Educator Protocol):
  • Section A (Background): Regional site, program type (CAPS, GREEN, After-School), years using program, school/CBO setting, grade level taught, course subject, low-income/free-lunch proportions, institutional awareness, administrative support (1–5 scale), program resource utilization, and national office support ratings.
  • Section B (Student Impact): 19 items (a–s) assessing observed youth development outcomes on a 4-point categorical impact scale (Large Impact, Small to Moderate Impact, No Impact, Negative Impact).
  • Section C (Educator & Institutional Impact): 18 items (a–r) assessing changes in pedagogical practices, workload, classroom climate, community partnerships, and professional efficacy on a 3-point change metric (Increased, Decreased, No Change), followed by qualitative appraisal prompts.
• Scoring and Quantification Rules:
  • Student Core Items (1–21): Scored on a 4-point response metric: Very True = 4, Sort of True = 3, Not Very True = 2, Not True at All = 1. No reverse-scored items are included in this section. Mean domain composite scores are calculated across respective subscales.
  • Retrospective Skills Matrix (Item 35, a–l): Scored: Not at all = 1, A little = 2, Pretty well = 3, Very well = 4. Difference scores are computed as $\Delta = \text{Score}_{\text{Now}} – \text{Score}_{\text{Beginning}}$, yielding an index of self-assessed developmental progression that avoids traditional pretest response-shift bias.

11. Permissions & Fee and Test Year

The Earth Force Survey evaluation batteries were formalized in the late 1990s and refined in the early to mid-2000s (with notable versions formalized in 2003–2005) under the research direction of RMC Research Corporation for Earth Force, Inc. The instrument was supported by federal and foundation grants, including investments from the Corporation for National and Community Service (CNCS) and the National Science Foundation (NSF).

The survey instruments are considered program evaluation instruments in the public domain for research and non-profit educational evaluation purposes, provided proper academic attribution is maintained. No commercial fee is required for non-profit educational institutions, researchers, or evaluators using the survey for scholarly inquiry or school-based program assessment. Commercial utilization, large-scale third-party redistribution, or inclusion in proprietary software packages requires formal permission from Earth Force, Inc. (Fairfax, VA) and RMC Research Corporation.

12. References

• Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
• Billig, S. H. (2000). Research on K-12 school-based service-learning: The evidence builds. Phi Delta Kappan, 81(9), 658–664. https://www.jstor.org/stable/20439767
• Flanagan, C. A., Syvertsen, A. K., & Stout, M. D. (2007). Civic measurement models: Tapping adolescents’ civic engagement. CIRCLE Working Paper, 55, 1–48.
• Howard, G. S. (1980). Response-shift bias: A problem in evaluating interventions with pre/post self-reports. Evaluation Review, 4(1), 93–106. https://doi.org/10.1177/0193841X8000400105
• Klindera, K., & Maza, N. (2001). Youth action: An evaluation of Earth Force’s CAPS program. RMC Research Corporation.
• North American Association for Environmental Education. (2010). Guidelines for excellence: Environmental education materials. NAAEE.
• RMC Research Corporation. (2004). Earth Force national evaluation report: Longitudinal outcomes in student civic and environmental engagement. RMC Research Corporation. http://cart.rmcdenver.com/instruments/earth_force.pdf
• Watts, R. J., Diemer, M. A., & Voight, A. M. (2011). Critical consciousness: Current status and future directions. New Directions for Child and Adolescent Development, 2011(134), 43–57. https://doi.org/10.1002/cd.310
• Zimmerman, M. A. (1995). Psychological empowerment: Issues and illustrations. American Journal of Community Psychology, 23(5), 581–599. https://doi.org/10.1007/BF02506983

13. Items of the Scale (Questionnaire)