Here is a fast way to find out whether a company-wide AI is safe to turn on. Have two people ask it the same thing. “What is the comp band for a staff engineer?” One of them works in People. One is a contractor three weeks into a three-month engagement. If they get the same answer, you do not have a memory layer. You have a leak with a calendar invite.
”It respects your permissions” is doing a lot of work
Most enterprise AI tools say they honor your existing access controls, and they are telling the truth. The catch is what that sentence means. The assistant inherits whatever the person asking can already reach. Microsoft says this plainly in its own documentation: if a user can open a confidential file, Copilot can read it and fold it into whatever it writes back.
That would be fine if company permissions were tidy. They are not. One analysis found a deployed Copilot brushes up against roughly three million sensitive records per organization, and that more than fifteen percent of business-critical files are exposed through oversharing and stale permissions. Almost none of this is malicious. It is a document library set to “everyone in the organization” two years ago by someone who has since left. The AI did not break in. It read what was already too open and repeated it out loud, faster than anyone could notice. Enough teams have been burned that the US Congress banned staff from using Copilot over exactly this.
Access has to ride along with the fact
The instinct is to filter the output: let the model see everything, then scrub the answer on the way out. That does not work. Once a model has read a restricted fact you cannot make it un-read it, and you cannot reliably catch every place it might surface in a summary. Permission has to be part of the data itself, enforced in the query before the model sees a single word. You scope retrieval to what this person is allowed to see, and the model only ever reasons over that slice.
Most-restrictive wins
A good answer is often stitched together from several sources. When that happens it should inherit the strictest permission among them. Synthesis can connect and shorten, but it can never widen access. If you could not see the underlying source, you do not see the summary built from it, and you do not find it in the citations either. That last part matters more than it sounds. Plenty of systems hide the restricted text and then cheerfully cite it by name.
The honest no
The other half of safety is knowing when to stop. When something is not yours to see, the right response is to say so, not to produce a confident guess that happens to land close. “That is limited to the People team, and I will not guess” is not a failure state. It is the feature, and it is one you can measure.
Two people, one question, two different answers, because they are cleared for different things. That is not an edge case to bolt on later. It is the thing that decides whether you can point an AI at your company’s knowledge at all.
