Home📖 About Aggi ARIA🚀 Launch Aggi ARIA📖 About the AI Health Check🚀 Launch the AI Health Check✅ AI Review Readiness Check🔑 AI Agent Inventory Check🧰 All free toolsInsightsAbout UsWays to work with usDr. GollaContact Us →
Aggi ARIA (Aggi Responsible Intelligence Assessor) · enterprise continuous compliance·Aggi ARIA AI Health Check (ARIA Shape B) · self-serve, 20 runs a day
Plain answers

My AI worked fine last month. Why is it different now?

Because almost everything around the model changes even when the model itself does not. New documents, an edited instruction, a new tool, your provider's own update; any of these changes the answers you get. Nobody sends you a notice, so the change usually arrives as a customer complaint.

Four things move under your feet

People assume the model was swapped. Usually it was not. These are the common causes, roughly in order of how often we see them.

  • ✓Your own content. A document was added, edited or removed, and the assistant now retrieves something different when it answers.
  • ✓Your instructions. Somebody improved the system prompt. One sentence in there can change the tone and the limits of every answer.
  • ✓Your tools. A new integration, a changed field name, an API that now returns something slightly different.
  • ✓The provider. Model versions are updated, and defaults change. Your account may move to a newer version without you asking for it.

Drift is quiet by nature

A failure is loud. Drift is not. The answers stay fluent and confident, the format looks the same, and only the substance moves. So the first signal is usually a person, not a dashboard: a customer who got told the wrong thing, or a colleague who says it used to handle that better.

That is the argument for checking on a schedule rather than when you get suspicious. A weekly run against the same questions makes a change visible as a change, rather than as an anecdote you cannot quite pin down.

Re-run the same questions and compare

Keep a fixed set of questions with the answers you expect. Run them on a schedule. Compare today against last week, not against your memory. When something moves, you will usually find the cause in the list above within a few minutes.

How often depends on how much changes around your model. If your content and instructions are stable, weekly is generally enough; if they are edited frequently, daily is better. Either is better than finding out from a customer.

Other things people ask

Did my model actually change?

Possibly, but it is the least likely explanation. Check your own documents, instructions and integrations first; those change far more often than model versions do.

Is this the same thing as hallucination?

No. Hallucination is inventing something that was never true. Drift is behaving differently than it used to. A system can drift without hallucinating, and hallucinate without drifting, so it is worth testing for both.

How would I notice early?

Only by having something to compare against. A fixed set of questions, the answers you expected, and a record of what came back each time.

Aggi ARIA Shape B runs these checks for you

Point it at your own model, run the tests on demand or on a schedule, and get a score and a report you can hand to someone else. Scheduled runs are the point of it; $30 a month covers up to four models and twenty runs a day.