Highlights • A tool to assess alertness behavior in a high-risk emergency setting is proposed. • Psychometric tests of the sleep, fatigue, and alertness behavior survey are positive. • The SFAB may be used to assess impact of fatigue mitigation interventions.

Abstract Background Workplace safety is a recognized concern in emergency medical services (EMS). Ambulance crashes are common and injury rates exceed that of the general working public. Fatigue and sleepiness during shift work pose a safety risk for patients and EMS workers. Changing EMS worker behaviors and improving alertness during shift work is hampered by a lack of instruments that reliably and accurately measure multidimensional beliefs and habits that predict alertness behavior. Objectives We sought to test the reliability and validity of a survey tool (the sleep, fatigue, and alertness behavior survey [SFAB]) designed to identify the cognitions of EMS workers concerning sleep, fatigue, and alertness behaviors during shift work. Methods We operationalized the integrative model of behavioral prediction (IMBP) and developed a pool of 97 candidate items and sub-items to measure eight domains of the IMBP. Five sleep scientists judged the content validity of each item and a convenience sample of EMS workers completed a paper-based version of the SFAB. We retained items judged content valid by five sleep scientists and performed exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and tests of reliability and internal consistency. We identified a simple factor structure for each scale and calculated means and standard deviations for each item and scale. Results We received 360 completed SFAB surveys from a convenience sample of 800 EMS workers attending two regional continuing education conferences (45% participation rate). Forty-seven candidate items and sub-items/options were removed following content validation, EFA, and CFA testing. Analyses revealed a simple factor structure for seven of eight domains and a final pool of 50 items and sub-items/options. Domains include: attitudes, normative beliefs, knowledge, salience, habits, environmental constraints, and intent. EFA tests of self-efficacy items failed to identify a simple factor structure. We retained two self-efficacy items based on Spearman–Brown correlation of 0.23 (p <0.0001). Conclusions Measurement of sleep, fatigue, and alertness behavior among EMS workers is challenging. We describe the development and psychometric testing of a survey tool that may be useful in a variety of applications addressing sleep, fatigue, and alertness behavior among EMS workers.