Aggi ARIA Shape B is $30 a month for one batch, which covers up to four models and twenty runs a day, with a seven-day money-back guarantee. You pay your own model provider separately for the calls the tests make, and a test run uses a small number of tokens.
Ours is the simple one. $30 per batch per month, a batch being up to four models and twenty runs a day, stepping down toward $24 as you add batches. Seven-day money-back guarantee, so the worst case is a week and thirty dollars.
Your provider's cost is the one people forget to ask about. Tests are calls, calls cost tokens, and those are billed to you by whoever runs your model rather than to us. We never pay for them and we never mark them up. The good news is that a behavioral test set is small by the standards of ordinary usage; a run is a few dozen short exchanges, not a workload.
Because the token cost is yours, bound it where it is billed. Provider dashboards allow a spending limit per key, and a modest cap on a key used only for testing is a sensible precaution.
We would like Shape B to show you the tokens a run consumed and to let you set a ceiling inside the product. That is not built today, and saying so is more useful to you than implying otherwise. Until it exists, the provider-side cap is the control that works.
A developer can write a first set of tests in an afternoon. The recurring work is scheduling them, keeping the history, comparing runs, and maintaining the set as your product changes. An hour of competent developer time generally costs more than a month of Shape B, which is the comparison worth making honestly rather than a feature list.
If you have the time and would enjoy owning it, build it. The build-versus-buy page sets out both sides without a thumb on the scale.
Add another batch. Each batch is another four models and another twenty runs a day, and the per-batch price steps down as you add them.
Per month. One batch at $30 covers up to four models and twenty runs a day, so the cost does not move with how often you test inside that allowance.
There is a seven-day money-back guarantee, which is why we would rather you tried it than read about it.
Point it at your own model, run the tests on demand or on a schedule, and get a score and a report you can hand to someone else. Seven-day money-back guarantee, and you keep control of your own provider spend.