Your new CSA score (hopefully without wrinkles)

The FMCSA’s new CSA (Compliance, Safety, Accountability) scoring system will forgive you for a little bad luck or a few failures in a particular area – maintenance, for example. But if you mess up in multiple areas – say maintenance, hours of service, and unsafe driving – you’re in trouble.

That new system is based on IRT (Item Response Theory), which is used in, but not restricted to, testing and statistics. It will replace SMS (Safety Measurement System) that currently drives CSA results. The change will happen in September 2019 if all goes according to plan.

We’re not supposed to know this since the FMCSA has not officially announced any changes. But information revealed in industry forums, symposiums, public documents and trade articles tells the story. Could the official announcement when it comes tell a different story? Sure. But it isn’t likely.

The new IRT scoring system is supposed to kick in next year. In fact, the FMCSA is running IRT alongside SMS, the old system, now. They want to iron out as many wrinkles as possible before going live. But everybody, including the FMCSA, understands no matter how good the rollout is, there will be more wrinkles. Maybe lots of wrinkles. Wrinkles in SMS are what caused this change in the first place.

Problems plagued CSA

CSA launched in 2010 with high hopes. OOIDA was skeptical, but the ATA was happy with it. That didn’t last long. In short order, as CSA results became available, its shortcomings were obvious. Drivers hated the fact that any accident would count against you even if you weren’t at fault. So would any citation, even if you were found innocent. Carriers didn’t like the percentile ratings that showed scores in comparison with other carriers. They could be perceived as less safe than they actually were.

But most of all, carriers disliked the SMS that could arrive at bad conclusions. How bad?

In one infamous case, the research arm of the ATA, the American Transportation Research Institute, found a negative relationship in the scoring between two categories, driver fitness on one hand and controlled substances and alcohol on the other.

“They’re negatively correlated,” Steve Bryan, executive vice president and general manager of Greenwood Village, Colo.-based SambaSafety, told Fleet Owner magazine. “If you followed that logic, you should drink and smoke dope and get behind the wheel of a truck.”

Among other things, Byran’s company develops software that mimics the CSA scoring system – whatever it is – so its carrier customers can track CSA scores in-house. Bryan is a longtime critic of SMS and, as of now at least, a booster of the new IRT system.

The industry appealed to FMCSA to make changes in the SMS system. But FMCSA held firm. Apart from some insignificant tweaks, no changes.

But in 2015, Congress got involved. As part of a larger transportation bill, lawmakers forbid FMCSA from publishing some CSA material on its website, including percentile ratings. They ordered a review of CSA by the National Academy of Sciences. NAS made a number of findings and recommendations. In a report issued in June of 2017, NAS said CSA was pretty much OK except for SMS. They recommended replacing it with IRT. A follow-up report by the NAS in June of this year said the project to do just that was on track.

So what does all this mean?

According to some, when the retooled CSA kicks in the hated percentile ratings will be gone. That would be a good thing, though according to NAS documents it is not a done deal. Another good thing: the new CSA will correct for the differences in state enforcement levels. Good carriers in tough enforcement environments won’t find themselves rated below less safe carriers in more easy-going areas. The NAS also recommended that the CSA correct for the risks of weather in some regions – snow in the Northwest, for example. Again, these are not verified facts.

Otherwise, there will be little obvious change. CSA will continue to evaluate carriers in the seven categories referred to as BASIC (Behavior Analysis and Safety Improvement Categories) criteria. Those categories are unsafe driving, crash indicator, HOS compliance, vehicle maintenance, controlled substances/alcohol, hazardous materials compliance, and driver fitness.

A bad CSA score (low is good, high is bad) will still mean an intervention of the part of FMCSA. An intervention can be anything from a warning letter to a full-blown audit. No change there.

But the basic focus of CSA will change. Rather than predicting the likelihood of accidents, IRT is intended to work toward prevention – a subtle distinction with not-so-subtle effects. CSA will focus on what is being referred to as the “culture” of a carrier.

Is their confidence in the new plan?

However, faith in that magical math is flagging. The Owner-Operator Independent Drivers Association has been quick to point out that the entire basis of CSA has been focused on compliance. With hundreds of regulations captured within the system, that is not likely to change. So while scores and rankings may reflect fondly on compliance-driven operations and their safety “cultures,” the endgame question on identifying crash risk remains to be seen.

I really think “attitude” is a better word. FMCSA wants you to be thinking about safety all the time in all aspects of your work and business. Maybe the best word is “common sense.” OK, that’s two words, but hasn’t full-time safety been the best practice all along?