Agent Security and Governance: A Control Framework for Enterprise Agent Deployment
A six-layer control framework for agent security and governance: identity, tool grants, data boundaries, audit logs, approval gates and lifecycle.
Agent security and governance covers the controls that constrain what an agent can do with its access, not just what it says in its output. Output risk is a harmful or wrong answer; action risk is an agent that deletes a table, moves funds, or reaches a system it was never granted. This framework draws on OWASP, NIST, Microsoft and IMDA material, and it presents no customer outcome data, no certification claim and no vendor performance figure.
What agent security and governance means, and how it differs from LLM output safety

Agent security and governance is the set of controls that decide what an agent is permitted to execute, and it is a different discipline from LLM output safety. OWASP's AI Agent Security Cheat Sheet splits that control surface into nine sections, among them tool security and least privilege, memory and context security, human-in-the-loop controls, and monitoring and observability (OWASP AI Agent Security Cheat Sheet, 2025–2026). Output safety governs what the model returns to a reader; agent security and governance governs what the system is allowed to do.
OWASP LLM06:2025 Excessive Agency names the action-side failure: damaging actions performed in response to unexpected, ambiguous or manipulated model output, caused by excessive functionality, excessive permissions or excessive autonomy (OWASP LLM06:2025 Excessive Agency, 2025). The entry separates this from insecure output handling, the output-side control.
The standards position is that existing practice carries over only partly. NIST's summary analysis of responses to its RFI on AI agent security, NIST 800-5, reports that commenters "widely agreed that AI agents present novel security threats" and that fundamental cybersecurity principles "will require adaptation to satisfactorily address agent security" (NIST 800-5, 18 May 2026). The governance baseline predates this: NIST AI RMF 1.0 treats GOVERN as a cross-cutting function infused through the other functions across the full product lifecycle (NIST AI RMF 1.0, January 2023).
要点: Agent security and governance is six separable layers, not one setting: identity, tool grants, data boundary, logging, approval, and lifecycle. Hardening one layer while the others sit at defaults does not reduce action risk.
How do I give an agent least privilege — identity, scoped credentials and per-action permissions?

Least privilege for AI agents starts with a unique identity per agent, not a shared service account. Microsoft's Zero Trust guidance for agentic risk states that organizations should "assign unique, auditable identities to agents to enable authorization, policy enforcement, and traceability," apply least privilege "only for the current task," and "deny everything else by default" (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026).
The identity model determines what a token can reach. An agent can act on behalf of a signed-in user (delegated), as its own agent identity (application-only), or through its own agent user account, and the token subject differs in each case; policies can target agent identity blueprints so they apply to existing and future agents created from that blueprint, and in the on-behalf-of flow the token exchange is evaluated by Conditional Access (Microsoft Entra Conditional Access for agents, current revision).
Credentials should be scoped to the tool, not to the user. The MCP specification version 2025-06-18 requires authorization servers to implement OAuth 2.1, clients to implement PKCE, and clients to implement Resource Indicators for OAuth 2.0 as defined in RFC 8707 so the token explicitly names the target resource, with the resource parameter identifying the canonical URI of the MCP server (MCP Specification 2025-06-18, Authorization, 2025-06-18). The same specification carries dedicated sections on the confused-deputy problem and token passthrough, which are the two ways a correctly issued token ends up used against the wrong resource.
OWASP's cheat sheet adds the per-tool scoping rule: grant the minimum tools required for the specific task, and scope each one by read-only versus write and by specific resource (OWASP AI Agent Security Cheat Sheet, 2025–2026). The Berkeley CLTC Agentic AI Risk-Management Standards Profile requires role-based permission management with granular permission boundaries over agent capabilities and resource access, explicit permission declaration, and real-time permission requests for high-risk actions (Berkeley CLTC Agentic AI Risk-Management Standards Profile, v1.0, February 2026).
The failure mode is permission-class collapse. A team grants read on two tables, then adds a write to fix a workflow, then reuses the same role for a second agent, and the effective permission set is no longer describable in one sentence. Microsoft names the result agent sprawl: unmanaged or over-permissioned agents that proliferate and reduce IT oversight, mitigated by lifecycle governance covering registration, approval, expiration and decommissioning (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026). The CSA/Token Security survey puts a number on the tail of that process: only 21% of organizations have formal agent-decommissioning processes, and agents "linger long past their intended use, retaining permissions and credentials" (Cloud Security Alliance, Autonomous but Not Controlled, fieldwork January 2026).
How do I isolate agent tools and execution so a compromised agent cannot reach the network or shell?
Tool isolation for agents starts by removing open-ended execution from the tool list entirely. OWASP's LLM06:2025 entry names extensions that "run a shell command" or "fetch a URL" as the open-ended cases to avoid, and requires extensions to execute in the authorising user's security context rather than the agent's own (OWASP LLM06:2025 Excessive Agency, 2025). The tool ecosystem is the second problem. A Queen's University study of open-source MCP servers found that 7.2% of analysed repositories contained at least one security vulnerability, credential exposure was the most prevalent pattern at 3.6%, 5.5% showed MCP-specific tool poisoning, and 66% contained at least one critical or blocker-level code smell (MCP at First Glance, Queen's University, dataset cut-off 1 June 2025). The sample basis matters, or the 66% gets misread: 1,899 repositories were mined, 583 analysed, and dynamic scanning covered 73 after 10 of the first 83 failed.
⚠️ Warning: 66% refers to code smells, not "66% of servers are exploitable." Vulnerability figures rest on 583 analysed repositories, and dynamic scanning covered only 73. Cite the sample basis alongside the number.
Exposure is measurable at internet scale. A July 2025 internet-wide scan identified at least 1,862 MCP servers exposed to the public internet answering unauthenticated tool-listing requests (Cloud Security Alliance, MCP Security Crisis, published 4 May 2026). Tool listing is itself an attack surface, which is why a named-tool allowlist with deterministic parameter validation does more work than a blacklist of known-malicious tools. A reported production incident shows the failure mode, and its cause was not excessive permissions. The Sysdig Threat Research Team documented what it assesses as the first documented case of agentic ransomware, dubbed JADEPUFFER, entering via CVE-2025-3248, a missing-authentication flaw in Langflow's code-validation endpoint allowing unauthenticated arbitrary Python execution, then running a destructive database-extortion playbook against the victim's production database server (Sysdig, JADEPUFFER, 1 July 2026). The control failure here was unauthenticated exposure, not over-broad authorisation. Isolation boundaries also land on specific capabilities: when an agent may generate and execute queries, the real risk is often semantic overreach in the generated statement rather than network reach, a boundary discussed further in the limits of LLM-generated SQL.
在 Singdata(本指南的发布方),我们这样落实这一点: 我们的 V2 架构把 V1 的表级/行级访问控制升级为生产级沙箱(Production-Grade Sandbox),让代理的工具调用在受控环境中执行,而不是直接触达底层系统。它覆盖的是执行隔离这一步,并不保证代理的权限配置、数据边界或审批流程本身正确,这些仍需读者按上文逐项设计并留存证据。
What data boundaries must I enforce — row and column scoping, tenant separation, retention and egress?
Data boundaries for agents must be enforced below the prompt, because a boundary expressed only in the system prompt can be restated by a manipulated input. Microsoft's guidance treats all external inputs, including retrieved content and tool outputs, as untrusted by default, and requires strict separation between instructions, data, memory and tool parameters (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026).
Row and column scoping belong in the query layer, not the agent's instructions. Berkeley's profile requires layered access controls for system resources and different APIs, and states that agents should be granted the minimum permissions required for intended tasks and functions (Berkeley CLTC Agentic AI Risk-Management Standards Profile, v1.0, February 2026). Tenant separation follows the same logic: the agent's identity should resolve to one tenant's data scope, and a cross-tenant read should fail at the data layer rather than be declined by the model.
Retention has no verified numeric standard in this source set. The requirement is qualitative: define, document and follow a retention period proportionate to the risk and the legal obligation, and apply it to memory and long-lived context as well as to logs, since Microsoft identifies long-lived context as a factor that increases leakage risk (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026).
Egress control is the boundary most often left implicit. Microsoft's mitigation set includes classifying and governing sensitive data with deterministic rules for use, retention and output, and monitoring and filtering outputs and logs to detect unauthorized disclosure (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026). Deterministic means enforced outside the model: an allowlist of destinations, a row limit, a column mask, a query timeout.
What must an agent audit log contain so I can replay a decision, and how do I make it tamper-resistant?
An agent audit log must let a reviewer reconstruct which agent, acting under which identity, called which tool with which parameters against which data source, and what came back. The OpenTelemetry GenAI semantic conventions define the spans to build that from: create_agent, invoke_agent, invoke_workflow, plan, and execute_tool, with agent identity carried on gen_ai.agent.id and gen_ai.agent.name, the tool on gen_ai.tool.name, the operation on gen_ai.operation.name, and the session on gen_ai.conversation.id (OpenTelemetry GenAI semantic conventions, status Development). Because the conventions are at Development status, track the current version rather than treating the attribute list as frozen.
Regulated deployments add fields. EU AI Act Article 12 requires high-risk systems to technically allow automatic recording of events over the lifetime of the system, and for Annex III point 1(a) systems the log must include the start and end time of each use, the reference database checked, the input data that produced a match, and the identification of the natural persons involved in verifying the results (EU AI Act Article 12, Record-Keeping, Regulation (EU) 2024/1689). Those obligations attach to high-risk systems from 2 December 2027 for Annex III and 2 August 2028 for Annex I, per Article 113(c), and the dates should be re-confirmed against the Official Journal before you rely on them.
The chat transcript is not an audit log. It records what was said, not which tool ran under which identity, and it cannot answer the question an auditor asks first: what was this agent permitted to do at the moment it acted. Microsoft's intelligibility pillar requires accessible post-execution logs that record actions, tools and outcomes for audit and incident response, and its human-oversight pillar requires reliable system-level mechanisms to pause or stop agents safely (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026).
Tamper resistance follows from where the log is written. NIST SP 800-53 Rev. 5 defines the Access Control, Audit and Accountability, and System and Information Integrity control families, with AU-02 event logging and SI-02 system monitoring updated in the 5.2.0 release of 27 August 2025 (NIST SP 800-53 Rev. 5, Rev. 5, September 2020). The agent should not hold write or delete rights on its own log stream, and the log should be written to a store the agent's identity cannot administer.
Which agent actions require human approval, and how should approval, timeouts and escalation be designed?
对不可逆或高影响操作设置智能体操作的人工审批,关键在于审批必须是真正的闸门,而不是事后通知。当智能体越出授权范围时,38% 的受访组织要求人工审批,24% 仅要求记录日志,只有 11% 会自动阻断该操作(Cloud Security Alliance 与 Token Security 联合调查,2026 年 1 月实地调研,2026 年 4 月 21 日发布;该样本由厂商委托,418 名受访者)。也就是说,记录越权行为后继续执行,是这批样本中最常见的处理方式。
审批设计包含四个部分。触发条件:明确列出始终需要审批的操作类别,并让触发逻辑由确定性规则判定,而非交给模型自行分类。超时处理:审批请求若始终无人响应,必须默认拒绝而非默认执行,否则闸门就退化成延迟。升级路径:审批人不可用时通知谁,智能体在此期间做什么。审计记录:审批人身份与决定必须写入日志。
提示: 在智能体上线前就确定超时与升级路径。上线后再补,往往意味着默认行为是继续执行。
代价是延迟。审批闸门会给原本设计为无人值守的流程加入人工响应时间,这种摩擦是真实存在的。IMDA 的《智能体 AI 模型治理框架》主张在重要检查点触发人工审批,并要求定期审计审批机制的实际有效性(IMDA Model AI Governance Framework for Agentic AI,版本待确认)。欧盟《人工智能法案》第 14 条则要求监督措施与风险、自主程度和使用场景相称,被指派人员须能覆盖或撤销输出,并通过停止按钮或类似程序让系统进入安全状态(EU AI Act Article 14);该条同时提醒,被指派人员需对自动化偏见保持警觉。Berkeley CLTC 的风险管理标准概要则把监督强度与自主等级挂钩,针对高风险操作提出实时权限请求的做法(Berkeley CLTC Agentic AI Risk-Management Standards Profile,v1.0,2026 年 2 月)。
How do I tier agent actions by blast radius and choose between on-prem, private cloud and SaaS?
Agent risk tiers should be assigned by blast radius, meaning the scope of damage if the action executes wrongly, and the tier should determine the control set. A read of a masked column in one tenant is a different tier from a delete against a production table, and the two should not share a permission role.
Tier | Action class | Typical controls |
|---|---|---|
1 | Read, scoped to named resources, masked columns | Per-tool read-only scope, logged, no approval |
2 | Create or modify in a non-production or reversible scope | Scoped write, logged, post-hoc review |
3 | Modify production data, send external communications, move funds | Approval gate, deterministic trigger, timeout defaults to deny |
4 | Delete, administer permissions, execute code, change configuration | Approval plus dual control, no autonomous execution |
The tiering logic has a standards basis. Berkeley's profile names unauthorized access and privilege escalation as a risk-tolerance consideration, and its Manage 1.3 control states that for malicious misuse organizations should enforce least privilege for tool access, secure delegation mechanisms, and segment complex tasks to limit the impact of a single compromised agent (Berkeley CLTC Agentic AI Risk-Management Standards Profile, v1.0, February 2026). Segmentation is the tier boundary in practice.
Deployment model changes who owns which control, not whether the controls are needed. Under Microsoft's shared responsibility model, across on-premises, IaaS, PaaS and SaaS the customer always retains data classification and protection, endpoints, accounts, and access management including RBAC, MFA and conditional access; the provider owns the physical datacenter, network and hosts in IaaS, PaaS and SaaS and the operating system in PaaS and SaaS; network controls are customer-owned on-premises and in IaaS, shared in PaaS, and provider-owned in SaaS (Microsoft shared responsibility model, current revision). Identity and users remain customer-owned in every model, which means agent identity governance does not transfer to a vendor at any tier.
The practical reading: a SaaS agent platform removes patching and host hardening from your scope and leaves you the identity, data boundary, approval and logging layers. An on-premises deployment returns host and network controls to you and adds the operational burden of running them. The tier table above applies unchanged in all three models, which is why the deployment decision should follow the tiering decision rather than replace it.
Checkpoint checklist for a pre-launch agent review
Use this list as the review artifact. Each item should have a named owner and a piece of evidence, not a verbal confirmation.
Identity. Every agent has a unique, auditable identity, and no two agents share a credential. Evidence: identity inventory export.
Permission scope. Each agent's effective permission set is written down, scoped per tool by read versus write and by named resource, and the denied classes are listed. Evidence: permission-scope document.
Default deny. Tools, data sources and operations not explicitly granted are denied by default. Evidence: configuration excerpt.
Tool inventory. All models, tools, plugins and data sources the agent can reach are inventoried and reviewed as part of the security boundary. Evidence: dependency inventory.
No open-ended execution. No shell, URL-fetch or arbitrary code tool is reachable by the agent. Evidence: tool list.
Data boundary. Row, column and tenant scoping are enforced in the query layer, and a cross-tenant read fails at the data layer. Evidence: test result.
Retention. A retention period is defined, documented and applied to memory, context and logs. Evidence: retention policy.
Egress. Destinations, row limits, column masks and timeouts are enforced outside the model. Evidence: egress allowlist.
Audit log. The log records agent identity, tool, parameters, data source, outcome and approver identity, and the agent cannot write to or delete its own log stream. Evidence: log schema and access policy.
Approval gates. Tier 3 and Tier 4 actions require approval, the trigger is deterministic, and a timeout resolves to denial. Evidence: approval configuration.
Stop control. A system-level mechanism can pause or stop the agent and bring it to a safe state. Evidence: tested stop procedure.
Lifecycle. Registration, approval, expiration and decommissioning are defined, and a decommissioning date exists for every agent in production. Evidence: lifecycle record.
Autonomy level. The agent's autonomy level is stated, and the oversight measures match it. Evidence: risk-tier assignment.
FAQ
What is the difference between agent security and LLM output safety?
Output safety governs what the model returns; agent security governs what the system is permitted to execute. OWASP LLM06:2025 defines excessive agency as damaging actions performed in response to unexpected, ambiguous or manipulated model output, and distinguishes it from insecure output handling (OWASP LLM06:2025 Excessive Agency, 2025). A team can pass every output filter and still have an agent with delete rights on production.
How common are AI agent security incidents?
In a survey of 418 IT and security professionals, 65% of organizations reported at least one AI-agent-related incident in the past 12 months, 61% reported data exposure, and 82% discovered previously unknown agents in the past year, while 68% nonetheless claimed strong visibility (Cloud Security Alliance, Autonomous but Not Controlled, fieldwork January 2026). A separate survey of 750 senior technology leaders found 54% experienced or suspected an incident and 34.9% confirmed one (Gravitee, The State of AI Agent Security 2026, fieldwork April 2026). The two surveys ask different questions, so read them as directionally consistent rather than directly comparable.
Do we need a separate identity for every agent?
Yes. Microsoft's guidance is to assign unique, auditable identities to agents to enable authorization, policy enforcement and traceability, and to apply least privilege only for the current task (Microsoft Zero Trust agentic risk guidance, updated 19 March 2026). Shared service accounts make the audit log unable to answer which agent acted, and they make revocation a cross-team project instead of a single action.
How long should we retain agent audit logs?
No verified numeric standard exists in the sources reviewed here, so the defensible answer is qualitative: define, document and follow a retention period proportionate to the risk and the applicable legal obligation. EU AI Act Article 12 requires high-risk systems to technically allow automatic recording of events over the lifetime of the system, which is a capability obligation rather than a duration (EU AI Act Article 12, Record-Keeping, Regulation (EU) 2024/1689). Set the period with your legal team and write it into the retention policy.
Does the EU AI Act apply to our agents today?
The high-risk obligations attach on 2 December 2027 for Annex III systems and 2 August 2028 for Annex I systems, per Article 113(c) (EU AI Act Article 14, Human Oversight, Regulation (EU) 2024/1689). Article 14 requires that high-risk systems be designed so they can be effectively overseen by natural persons, and the oversight measures must be commensurate with the risks, level of autonomy and context of use. Confirm the dates against the Official Journal, and note that the excerpt reviewed covers high-risk systems rather than all agentic deployments.
Can we rely on our existing application security tooling for agents?
Partly. NIST's summary of RFI responses reports that commenters agreed fundamental cybersecurity principles remain relevant but "will require adaptation" for agent security (NIST 800-5, 18 May 2026). The Queen's University study found that a traditional scanner detected no vulnerabilities in official MCP servers while an MCP-specific tool found tool poisoning in 5.5% of servers, and the authors conclude that traditional tools "only find part of the whole landscape" (MCP at First Glance, Queen's University, dataset cut-off 1 June 2025).
What is the biggest gap in most agent governance programs?
Scope-violation handling. When agents exceed their scope, only 11% of organizations automatically block the action, against 38% that require human approval and 24% that require logging, and only 21% have formal decommissioning processes (Cloud Security Alliance, Autonomous but Not Controlled, fieldwork January 2026). Logging a violation and continuing means the control detects but does not prevent.
Conclusion and next steps
The evidence supports one organizing conclusion: agent security is six separable layers, and the failure modes cluster where a team treats them as one. Identity, tool grants, data boundary, logging, approval and lifecycle each fail independently, and the survey data shows the pattern, with 82% of organizations discovering unknown agents and only 11% automatically blocking scope violations (Cloud Security Alliance, Autonomous but Not Controlled, fieldwork January 2026).
Two constraints should shape the plan. The first is that the control set is not yet standardized for agents: NIST's RFI summary reports agreement that existing practice needs adaptation, and the OpenTelemetry conventions that give the audit log its field names are still at Development status (NIST 800-5, 18 May 2026). The second is that the regulatory clock is real but not immediate, with the EU AI Act high-risk obligations starting 2 December 2027 and 2 August 2028 (EU AI Act Article 14, Human Oversight).
The next step is to run the checkpoint checklist against one agent already in production, not against the next one you plan to launch. Pick the agent with the broadest permissions, complete the identity, permission-scope and audit-log items first, and record the gaps you cannot close this quarter. That gives you a defensible position for an audit response and a concrete backlog for the review cycle.