Nobody had one trust level

In every institution I worked in, nobody was trusted. Particular things they could do were. It took me an embarrassingly long time to notice that the AI conversation had forgotten how to make that distinction.

Fourteen years in regulated financial services teaches you what authority actually looks like from the inside, and it does not look like a level of confidence in a person. It looks like a table. Someone can release this type of item, up to this size, during these hours. Above that number a second name is required. This category needs a different desk entirely. Access gets reviewed on a schedule, and the review is not a conversation about whether the person is good at their job — it is a line-by-line pass over what they are currently able to do and whether they still need to be able to do it.

Nobody in that world ever asked how much we trusted a colleague. The question didn't parse. You asked what they were entitled to, under which limit, with whose countersignature above it. Two people with the same title and the same tenure could have materially different entitlements, and neither of them took it personally, because the entitlement was never a statement about them. It was a statement about the action.

Authority was granted to the transaction, not to the person holding it.

Then the question changed shape

So when AI systems started arriving with real permissions attached, I kept hearing a question I had not heard asked seriously about a human being in my entire career: how autonomous is it?

One number. For the whole system. The same system that might read a submitted document, draft something internal, update a record, send a message to a client, change who has access to what, and move money — and the answer to all six is apparently a single setting on a configuration page.

In the institutions I came up in, those six things would not have belonged to one person. Several of them would not have belonged to one department. The one that moves money would have had its own control around it that everyone accepted as non-negotiable, not because anyone doubted the competence of whoever pressed the button, but because the action could not be taken back and the amount at stake did not care how good that person was.

That is the whole observation, and it is not a sophisticated one. We solved this for people a long time ago. We solved it badly at first, and then regulators and a few expensive incidents made us solve it properly, and the solution was always the same move: stop rating the actor, start classifying the act.

The part that surprised me

What I did not expect was that the useful output of doing this isn't the finished table. It's the argument you have while filling it in.

Anyone who has sat through an access review knows the shape of it. Someone from engineering says a change is easily undone. Someone from operations points out that three downstream teams will have consumed the value within the hour. Someone from risk raises a consequence nobody in the room had modeled, and the meeting stops being administrative. That disagreement is not friction in the process. It is the only place the real architecture of the thing becomes visible before an incident makes it visible for you.

I have started to think the same is true of AI deployments, and for the same reason. The grid is easy. Writing down every action the system can actually take, in front of the people who will each read that list differently, is the part that does the work.

The systematic version of this — the two questions to ask about every action, what each answer obliges you to build, and the objections it has to survive — is a longer argument, and I worked it into a full framework for the practice.