Third-party AI is your risk now

Between March 10 and March 16 this year, attackers were inside one of CareCloud's Amazon Web Services environments for six days.

CareCloud sells cloud software for the EHR (the electronic health record, the digital chart your doctor's office keeps on you), along with billing and practice management. More than 45,000 healthcare providers run on it.

In early July, state filings put the number of affected people at roughly 350,000. In August the federal breach tracker at Health and Human Services listed 3,371,508. The next day it read 3,756,469.

The exposed data included names, addresses, dates of birth, Social Security numbers, driver's license numbers, bank account and card numbers, and medical and health insurance information.

Almost none of those 3.7 million people have heard of CareCloud. They were patients of practices that chose a vendor.

Two things stay with me, and neither is about blame

First, four months passed between "we contained it" and "we know what left," and the count grew roughly tenfold along the way. I do not read that as negligence. Reconstructing an exfiltration, which means working out exactly which records left the building, is genuinely hard work. It does tell you how long the honest answer takes.

Second, there is a question I cannot put down. Data forensics is a mature discipline. It is slow and painful, but it works, because there are logs and artifacts, and eventually you can say what happened.

Now ask the same question one layer over.

If an attacker spent six days near your AI systems (the models, the prompt templates, the retrieval corpus they draw on), could anyone reconstruct whether the behavior changed? There is no exfiltrated file to recover. A model that starts answering differently leaves no artifact at all, unless somebody was measuring it beforehand.

Most organizations have no behavioral baseline

That is not carelessness; nobody told them they needed one.

So the honest question for anyone running vendor AI, meaning AI inside software you bought rather than built, is whether you have any way to tell whether its behavior changed last month, or whether that visibility stops at the vendor's front door.

We work in AI compliance, so we have a horse in this race. We raise it anyway, because we do not think the answer is comfortable for anyone right now, vendors included.

← Back to Insights