Has it made a difference?
The question most often answered with the wrong findings.
Warning sign: you can show sessions run, controls tested and tickets closed, but not one number that would have moved if the work had failed.
What I think
The test I use is simple. Take your headline measures and ask which of them would have gone down if the work had achieved nothing. If the answer is none of them, you're measuring activity.
What you can measure here splits in two, and the split is the finding. In the study I rely on (Dell'Acqua and colleagues, 2026), consultants using AI completed more tasks, faster and to a higher standard, on tasks the tool handled well. On a task just outside that, a separate group using the same tool was 19 percentage points less likely to be correct.
So it wasn't better or worse on average. It was much better on one side of a line and much worse on the other.
That line is uneven, varies by field and is hard to predict, so it's worth being precise about whose line it is. Yours is what you can judge. The tool's is what it was built to do well. The two aren't the same, and the gap between them is where confident work that nobody can check gets produced.
That's why "I can't judge this" is the most valuable thing a professional can say, and the hardest to say out loud.
One thing I want to be careful about. In the study, on tasks the tool handled well, the gains leaned towards the weaker performers, lifting them towards the average. My own view goes further: on harder work, the same tool can lift the expert and mislead the novice. That isn't a measured finding, because the study measured tasks rather than people, but I think the inference is right.
On the other side, failure is now much more visible. When someone is found to have put their name to work they didn't check, it tends to spread in a few predictable ways. It's permanent, because the record is searchable. It escalates, from the individual to the firm to the profession. And it passes to everyone downstream who relied on the work. These are the usual routes, and they don't always play out.
The public record of AI failures shows what came to light, which is only part of what happened. It leans towards law because legal work gets examined in public. That doesn't mean law has the most failures.
What we don't know yet
Whether this pattern repeats beyond the tasks that have been studied. It's a small evidence base carrying a lot of argument.
What a good measure of quality looks like when the usual failure is something missing. Most of what goes wrong in AI-assisted work is something that isn't there, and that's exactly what activity measures can't see.
As of September 2026, the strongest numbers on this page come from one study. I haven't found a second of the same quality, so there's still a lot of real-world experience to unpack.
What you can do
Before you hand a task to an AI tool, ask: if it gets this wrong, will I be able to tell? If not, slow down and do enough yourself to judge it, bring in someone who can, or don't use the tool for this one.
Take your headline measures and ask which of them would have gone down if the work had failed. Keep those. The rest measure activity.
Before the work goes out, write three lines: what you used, how you checked it, and what would make you pull it back.
If the answer to this question is no, go back to the first one, do we actually know what is happening?, before you change the measure.