Does it stick when nobody is watching?
Perhaps the question that matters most.
Warning sign: it worked while there was a change programme, then slipped within six months of the final report.
What I think
Most change models put keeping the change going last: the control step, the reinforcement step, the maintenance phase. It's the item that gets cut when the programme runs late. Treating it as a question in its own right is the one structural change I'd argue for hardest.
There are two versions of the problem, and they're coming together.
The human version is that a way of working can't simply be handed over in writing. People build it through practice and reflection. To last, it has to move from one person to a team, from the team to the organisation, and from there into something durable. Each step works differently: first people feeling safe to speak up, then spreading good practice with discipline, then recognition and reward for persisting, then things like checklists repeated as an accepted part of doing the work.
My reading, and it's my own synthesis rather than a finding, is that the essential parts have to survive every one of those handovers. What usually breaks is the judgement the tool was standing in for.
The machine version is newer, and here I'm reasoning rather than stating. As AI moves from helping with tasks to carrying them out on its own, responsibility moves with it. It shifts from checking each output to answering for the whole system: how it's designed, the limits it runs within, and the standard it keeps when nobody is looking.
What isn't in doubt is how fast this is moving. The length of task an AI agent can finish successfully half the time has been doubling roughly every seven months over six years. Since 2023, it has been doubling roughly every four months (METR, Time Horizon 1.1, January 2026). That measures what agents can do, not how reliably they do it.
Two ways this goes wrong are worth knowing. Over a long run, an agent follows its instructions less closely, so a system that was sound on day one may not be on day ninety. And the model only asks for an action, while other software carries it out. So an assistant can report a job as done when it didn't finish it, got stuck, or never started.
One thing I wouldn't do. The obvious response to AI doing junior work is to stop juniors doing it. The early evidence, mostly from surgery and medicine rather than office work, suggests that starves people of the practice that builds expertise. Carrying it over to knowledge work is my inference. The fix is to design the work so juniors still get that practice.
What we don't know yet
What answering for a whole system actually involves, when the system acts on its own. I can say that responsibility moves up. I can't yet define an answer, but we need to work out how to test whether quality has become embedded in the way a system works, rather than creating a parallel world of governance that becomes detached from achieving objectives.
What you can do
Take a task you finished this week and write it out as simple steps a colleague could follow without you. Then mark the step that used to be how a junior learned, and decide how they'll still get that practice. How would someone learn to do what you have just done?
Before anything runs without you watching, write three lines: the standard it must meet, the sign that it's drifting, and the event that makes you stop and look again. These are all based on the outcome you're looking to achieve.