FMCSA safety measurement system ‘lacks implementation details’
FMCSA’s corrective action plan for improving its Safety Measurement System and Compliance, Safety and Accountability program “lacks implementation details,” according to an audit from the U.S. Department of Transportation’s Office of Inspector General.
As part of the 2015 FAST Act, FMCSA was directed to commission the National Academy of Sciences to evaluate the methodology and data it uses to identify carriers that are not fit to operate commercial motor vehicles, and develop a corrective action plan in response. The legislation also directed the Office of Inspector General to assess FMCSA’s plan.
“While FMCSA’s corrective action plan addresses motor carrier safety interventions, it lacks implementation details for improving transparency and its assessment of carrier safety rankings,” the OIG report said.
FMCSA established the CSA program with the aim of reducing crashes of large trucks and buses by prioritizing higher-risk carriers for safety interventions. The agency uses the SMS algorithm to identify carriers for intervention by computing percentile rankings for each carrier in seven areas. Carriers with percentile ranks above FMCSA thresholds may receive interventions, which range from warning letters to onsite investigations.
In June 2017, the National Academy of Sciences issued a study on the program with six recommendations.
- Develop an Item Response Theory model over the next two years, and if it performs well in identifying and prioritizing motor carriers for intervention, use the IRT model to replace SMS.
- Collaborate with states and other agencies to improve the quality of data in the Motor Carrier Management Information System in support of SMS. Two specific data elements require immediate attention: carrier exposure and crash data.
- Investigate ways of collecting data that will likely benefit the recommended methodology for safety assessment, including data on carrier characteristics such as driver turnover rate, type of cargo, method and level of compensation, and better information on exposure.
- Structure a user-friendly version of the MCMIS data file without personally identifiable information to facilitate its use by external parties, such as researchers and carriers. Make user-friendly computer code used to compute SMS elements publicly available to individuals in accordance with reproducibility and transparency guidelines.
- Undertake a study to better understand the statistical operating characteristics of the percentile ranks to support decisions regarding the usability of public scores.
- Use both SMS percentile ranks and the SMS measures to prioritize which carriers receive alerts. Compute safety scores conditionally within groups of similar carriers, as well as unconditionally for all motor carriers.
The Office of Inspector General said FMCSA’s plan has failed to meet some of the recommendations.
“For example, in response to recommendations from the National Academy of Sciences and the Government Accountability Office, the agency is testing an Item Response Theory statistical model to gauge how it prioritizes motor carrier safety interventions,” the OIG report states. “Regarding the NAS recommendation on collecting more accurate and diverse types of data, FMCSA determined that much of the data do not exist. As a result, FMCSA no longer plans to collect additional data.”
According to the OIG, FMCSA officials said the agency is unlikely to collect the data before the IRT model is complete because of resistance from the motor carrier industry and the length of the regulatory process.
“FMCSA plans to assess the effectiveness of the IRT model without collecting more accurate and different types of data,” the OIG report said. “Thus, the agency will not have made the NAS-recommended improvements to ensure that exposure data are accurate and complete when it evaluates the IRT model, which could limit the model’s effectiveness.
“FMCSA did not provide alternative actions or implementation details, such as cost estimates and benchmarks for completion, for addressing NAS recommendations No. 3. This may limit the agency’s ability to improve its identification of high-risk motor carriers and mitigate future crashes.”
The Owner-Operator Independent Drivers Association said that any such program must have consistent and standardized data.
“The real driving force and what needs to be improved upon is the quality of data going into whatever system FMCSA is going to use to judge these carriers,” Jay Grimes, OOIDA’s director of federal affairs, told Land Line Now. “I think (the OIG report) certainly lets the FMCSA know it has some work to do.”
Related article:
Related News
