Now try to name one that tells you something already in your systems is wrong and should come out.

Most firms cannot, and the reason is not an oversight by any particular vendor. It is what the market asked for. Nobody buys software to reduce their own numbers, so nobody built it.

Why finding more is the easy half

The models underneath most business AI are built to predict. Given an input, estimate what probably applies, rank by confidence. That produces a ranked list of candidates, and a ranked list is useful for exactly one thing: surfacing items worth a human look.

What it cannot produce is a determination that an existing entry is wrong. There is no position in a ranked list that means “remove this.” You can score something low. You cannot score it as incorrect.

So your systems accumulate. The CRM fills with contacts who left their jobs in 2023. The supplier list keeps firms that no longer trade. The compliance folder holds certificates that expired. Everything that was ever added stays added, because nothing in the stack is responsible for taking things out.

What that costs at scale, measured

American healthcare has been running the extreme version of this, and federal auditors published the numbers.

In US Medicare Advantage, insurers are paid more for covering sicker patients, with illness established by diagnosis codes drawn from clinical notes. An industry of software grew up to read those notes and find conditions that had been missed. Finding more meant revenue, so finding more is what got engineered.

The US Office of Inspector General audits whether those diagnoses hold up. Across codes it considers high risk, it reports approximately 70 percent were not supported by the associated medical records, with some categories above 90 percent. One audit sampled 97 records carrying an acute stroke diagnosis. None were supported as acute strokes. In 68 of those cases the patient genuinely had had a stroke, documented in their history, and the code described it as currently happening.

Nothing was fabricated. The software did its job. Its job never included noticing that something already recorded had stopped being true.

The question that exposes this in a demo

Vendors demo the finding. They rarely demo the opposite, because there usually is not one.

Ask to see what the system produces when it disagrees with data you already hold. Not described, shown on screen. If the answer is a lower confidence score, that is a ranking, not a correction, and your records will keep growing in one direction regardless of what your policies say.

Two follow-ups worth having ready.

Can it mark something as unsupported, or only rank it below something else? Those are different outputs and only one is a control.
Who in our business is responsible for removing things, and what tool do they use? If the honest answer is nobody and none, that is the gap.

What a fix actually looks like

Systems that can say no work differently underneath. Rather than scoring how closely something resembles a pattern, they test it against criteria that are written down. Because the criterion is explicit, the test can fail, and a failed test is a finding you can act on.

That is the shift happening in the healthcare case above, where buyers are moving to a risk adjustment platform that flags unsupported records as well as missing ones. The sector got there because a regulator published the test and then ran it. Most businesses get to ask the question before someone else asks it for them.

Read the full article →