Do we actually know what is happening?
The question the other four depend on.
Warning sign: your evidence is a summary of a summary, and nobody can say who checked the original material.
What I think
Writing a report used to mean reading the file behind it, so a well-written report was a fair sign that someone understood the material. AI has broken that link. A tool can produce a polished report without anyone working through the substance, so polish no longer tells you much.
That leaves two questions to ask of any piece of work, and "is this correct?" isn't one of them. The first is how it was made: who did it, on what evidence, how the key decisions were taken, and why it's shaped the way it is. The second is who is standing behind it. That means a named person, competent to judge the work, who takes personal responsibility for it against a stated standard. If nobody can be named, nobody is standing behind it.
It matters now because what these tools can do and how reliably they do it have come apart. In the study I rely on most (Dell'Acqua and colleagues, 2026), consultants did measurably better on tasks the tool handled well. On a task just outside that, a separate group of consultants using the tool was 19 percentage points less likely to be correct.
Tools built for the job aren't exempt either. Legal research tools built for the purpose hallucinated 17% to 33% of the time (Magesh and colleagues, 2025). That means they gave a wrong answer, or backed an answer with a source that didn't support it. A separate study asked general-purpose models specific questions about randomly chosen US federal court cases. It found hallucination rates of 58% to 88% (Dahl and colleagues, 2024). The two studies measured different things, so you can't compare the figures directly.
The failures are usually quiet. In April 2025 Nature reported that science sleuths had flagged hundreds of published papers that used AI without saying so (Nature, 2025). In some cases, the telltale signs later disappeared without a trace.
One limit doesn't go away with better models. The tool only knows what it's given. You know things it never had: what the work is for, who will read it, and the facts nobody wrote down. When something is missing, the tool fills the gap with a confident guess in the same voice as everything else. Only you know what was never said, which is why this check can't be handed to the machine.
What we don't know yet
How much checking is enough for a given risk. The principle is easy: check in proportion to the cost of being wrong. Setting the level is harder, and I don't think one answer fits every field.
Who can stand behind work that crosses two fields when nobody is expert in both.
Whether a record of how work was made survives becoming routine. Every such record I've seen either stays a real practice or turns into a form people fill in. I don't know yet what keeps it in the first state.
How much undisclosed AI-assisted work is in circulation. As of September 2026 I have no reliable way to estimate it. The published cases are only the ones that came to light.
What you can do
Before you start a piece of work, write one short paragraph. Say what it's for (the outcome), what decision it supports, who will rely on it without checking it again, and what must be true for that to be safe.
Before you sign off an AI-assisted draft, check the two or three claims that would do most damage if they were wrong. Check them against the original source.
Write down what you checked and what you found, including what passed.