How We Score AI
- Niv Nissenson
- Apr 4
- 1 min read
Updated: Apr 9
At The Tester AI, every test is built around a simple principle: Can AI actually do the job—not just in theory, but in practice? Each test is evaluated across five core categories, scored on a scale of 1 to 5:
Output Delivered – Did the AI complete the task?
Accuracy – How correct was the result?
Quality – Is the output usable in a real-world setting?
Ease of Use – How much effort, prompting, or iteration was required?
Reliability – Was the behavior consistent or unpredictable?
The final score is a weighted average, reflecting overall real-world usability—not just whether the AI can do something, but whether it can do it well and consistently.
What the Scores Mean
We group results into three clear verdicts (results are rounded to the closest half unit):
1.0 – 2.0 → Fail
The AI cannot reliably complete the task or produces unusable results.
2.5 – 3.0 → Partial
The AI gets part of the way there but requires workarounds, fixes, or human intervention.
3.5 – 5.0 → Pass
The AI successfully completes the task with a usable, reliable output.
The Human Benchmark
Each test also includes a simple but important question:Can a human do it better?
The answer is can be yes or no but also a yes can have tradeoff effect (a human can do it better but at a much higher cost for instance).This comparison is critical. AI doesn’t need to be perfect to be useful it just needs to be good enough relative to the time, cost, and effort saved.
