While building Quest, I realized people often conflate two important facets of agentic permissions.
One facet is the systemic one. "Can the agent send a Slack message?" "Can the agent write to this file?"
We often think about these permissions in the context of a specific workflow or task. We build systems that say for workflow X, the agent can call function Y. We map the tools that an underlying system provides us into a permission layer for the AI.
Which is great; it's part of the solution, but if you stop there you're missing half the story.
Because the second facet is: "Even if the agent can write to this file, should it write what it's about to write?"
Here's where it gets way more nuanced. The agent just used your CFO's access permissions to read everyone's salary. Can it use that information to write it to a file? If it does, who can see that file? Is there a violation of privacy, confidentiality, or company rules if it does?
As you can see, just thinking about things at the "can it write to a file" level doesn't address a huge set of potential issues.
And unlike these tool-level access controls, the rules of information flow within an organization are often fluid, poorly documented, and highly contextual.
This is why in Quest, we went with a strict human-in-the-loop design for all scenarios where an AI can share information retrieved from your context with other people/agents.
Someday, we hope to introduce mechanisms to allow AIs to help you make these decisions. If these gatekeeper AIs get good enough, then you might imagine some of this even gets automated.
But for this to work there's a whole set of skills and context that needs to be built for every organization according to their own rules.
And then there's the whole layer of evals and testing needed to measure resilience against prompt injection and alignment behavior. All before we can ever hope to turn this on for handling the most sensitive data.
Until then, the agent can still do a lot of work for you, but I think you should have the final say.