Home📖 About Aggi ARIA🚀 Launch Aggi ARIA📖 About the AI Health Check🚀 Launch the AI Health Check✅ AI Review Readiness Check🔑 AI Agent Inventory Check🧰 All free toolsInsightsAbout UsWays to work with usDr. GollaContact Us →
Aggi ARIA (Aggi Responsible Intelligence Assessor) · enterprise continuous compliance·Aggi ARIA AI Health Check (ARIA Shape B) · self-serve, 20 runs a day
Plain answers

How do I know if my AI is making things up?

You find out by testing it, repeatedly, with questions where you already know the right answer. A model that invents an answer usually does it on a small number of predictable subjects, and the only way to see the pattern is to keep a record over time.

A single good answer proves very little

Ask a model the same question twice and you can get two different answers. That is not a fault in your setup; it is how these systems work. So a spot check on a Tuesday afternoon tells you how it answered once, on Tuesday afternoon. It does not tell you what your customers are seeing all week.

What does tell you something is repetition. Ask the same set of questions every day, write down what came back, and look at the pattern. Invented answers are rarely evenly spread; they cluster around a few subjects, usually the ones where your own documents are thin or out of date.

Three kinds of question, and you need all three

Most people test only the first kind, which is why problems reach customers.

  • ✓Facts you can verify. Your prices, your hours, your policies, the things a wrong answer would embarrass you about. You already know the right answer, so scoring is easy.
  • ✓Things it should refuse. Medical or legal advice, promises about refunds, anything you would not put in writing. A model that helpfully answers these is a liability.
  • ✓Rules it must never break. Never quote a discount. Never name a competitor. Never ask for a card number. Write them down as sentences, then check them.

The record is the product, not the test

A test you run once and forget is a feeling. A test you run daily, with the results kept, is evidence; and evidence is what lets you say when something changed, what changed, and whether you fixed it.

It is also what you show someone who asks. A customer, an insurer, a larger company doing vendor review. None of them wants your policy document. They want to see that you checked, and what you found.

Other things people ask

How often should I check?

For most small businesses, daily is comfortable and weekly is usually enough. The right answer depends on how often the things around your model change; if you update your documents or your instructions often, check more often.

Can I do this myself without buying anything?

Yes. Twenty questions in a spreadsheet, run by hand once a week, is a real improvement on nothing. The work that gets tiring is running them on a schedule and keeping the history, which is where a tool starts to earn its place.

What counts as a failure?

Whatever you decided before you ran the test. That is why writing the expected answers down first matters; without it, every answer looks plausible.

Aggi ARIA Shape B runs these checks for you

Point it at your own model, run the tests on demand or on a schedule, and get a score and a report you can hand to someone else. It is $30 a month for up to four models and twenty runs a day, with a seven-day money-back guarantee.