Highlights • A Bayesian propensity score analysis is compared to a frequentist propensity score analysis. • A no-treatment study design using observational data was used for the comparison. • The Bayesian nominally outperformed the frequentist method in the largest sample. • The methods produced near-identical results in the medium sample. • Neither method closely approximated assumed, true safety effect using small sample.

Abstract The propensity score matching method has been used to estimate safety countermeasure (treatment) effects from observational crash data. Within the counterfactual framework, propensity score matching is used to balance the covariates between treatment and control groups. Recent studies in traffic safety research have demonstrated the strength of this method in reducing the bias caused by treatment site selection. However, several general issues associated with safety effect estimates may still influence the effectiveness and robustness of this method. In the present study, Bayesian methods were integrated into the propensity score matching method. Bayesian models are known for their ability to capture heterogeneity and modeling uncertainty. This may help mitigate unobserved variable effects in the roadway and crash data. Furthermore, the sampling-based algorithm used for Bayesian estimation yields more consistent estimates in small region analysis than estimates from frequentist modeling. In this study, a dataset that was used to evaluate the safety effects of the dual application of shoulder and centerline rumble strips on two-lane rural highways was acquired. Only data from the before treatment period were used in a no-treatment effect analysis in order to compare the results of a Bayesian propensity score analysis to a frequentist propensity score analysis. Because no treatment was applied during the analysis period, it was assumed that there would be no treatment effect, or a crash modification factor equal to 1.0. The Bayesian propensity score matching method nominally outperformed the frequentist propensity score matching method in the largest sample and produced near-identical results in the medium sample, but neither method closely approximated the assumed, true crash modification factor in the small sample analysis. A simulation study is recommended to further study the effects of sample size and confounding factors when comparing the Bayesian and frequentist propensity score matching methods.