A novel variable selection method based on Frequent Pattern tree for real-time traffic accident risk prediction

Traffic accident data are usually noisy, contain missing values, and heterogeneous. How to select the most important variables to improve real-time traffic accident risk prediction has become a concern of many recent studies. This paper proposes a novel variable selection method based on the Frequent Pattern tree (FP tree) algorithm. First, all the frequent patterns in the traffic accident dataset are discovered. Then for each frequent pattern, a new criterion, called the Relative Object Purity Ratio (ROPR) which we proposed, is calculated. This ROPR is added to the importance score of the variables that differentiates one frequent pattern from the others. To test the proposed method, a dataset was compiled from the traffic accidents records detected by only one detector on interstate highway I-64 in Virginia in 2005. This data set was then linked to other variables such as real-time traffic information and weather conditions. Both the proposed method based on the FP tree algorithm, as well as the widely utilized, random forest method, were then used to identify the important variables or the Virginia data set. The results indicate that there are some differences between the variables deemed important by the FP tree and those selected by the random forest method. Following this, two baseline models (i.e. a nearest neighbor (k-NN) method and a Bayesian network) were developed to predict accident risk based on the variables identified by both the FP tree method and the random forest method. The results show that the models based on the variable selection using the FP tree performed better than those based on the random forest method for several versions of the k-NN and Bayesian network models. The best results were derived from a Bayesian network model using variables from FP tree. That model could predict 61.11% of accidents accurately, while having a false alarm rate of 38.16%.