Typed Decisions Are Not Trusted Decisions: Governing Enterprise Data Agents
Typing guarantees form, not trust. Five testable layers for governing enterprise data agents: correctness, calibration, permissions, audit.
A typed decision is well-formed, not trusted. A schema can guarantee that an agent's output parses and satisfies a contract; it cannot guarantee the decision is correct, authorized, calibrated, or reconstructable after the fact. That gap is the whole problem in governing enterprise data agents, and it is measurable: on the BIRD benchmark, a 2023 study spanning 95 databases, 33.4 GB of data and 12,751 text-to-SQL pairs across 37 professional domains, GPT-4 reached 54.89% execution accuracy against 92.96% for human experts. On the WebArena benchmark, also 2023, the best GPT-4 agent completed 14.41% of 812 long-horizon tasks against 78.24% for humans. A valid query can still answer the wrong question.
This guide separates five trust layers that a passing type check does not cover: what typing guarantees, whether a valid decision is correct, how confidence is calibrated and when an agent abstains, what permission-aware context requires at retrieval and tool-invocation time, and what a decision record must contain to survive an audit. It closes with the escalation threshold for human review.
After reading, you will be able to test your own agent against each layer and say which ones you have actually implemented. One limitation up front: the criteria below are an evaluation checklist, not a certification. Passing all five does not prove an agent is safe, and no framework in this space claims otherwise.
What does 'typed' actually guarantee about an agent's decision — and what does it not?
Typing guarantees that a decision is well-formed against a declared contract. It does not guarantee the decision is correct, authorized, or trustworthy, and the sources that define the contract are careful not to claim more.
TypeSafe's Jev 1.14 announcement, published October 14, 2026, is a useful test case because it never defines "typed" explicitly. Its closest statement is that the box type "returns a single bounding box - no type errors allowed." That is a statement about the interface, not about the decision. The same release places three capabilities behind what it calls the danger-zone flags, composable images, the box type, and longer context, and frames them as experimental rather than production-ready.
The NIST AI Risk Management Framework makes the general version of this point: it names seven trustworthiness characteristics, including valid and reliable, safe, and accountable and transparent, and states that "addressing AI trustworthiness characteristics individually will not ensure AI system trustworthiness." A type system that enforces one property is exactly that, one property.
For decision trustworthiness for AI agents, the practical reading is that a type check is a precondition, not a verdict. It tells you the agent produced something the rest of your pipeline can consume. Everything after that is a separate question, which is why consistent definitions make answers testable rather than merely parseable.
Key Takeaway: Typed Decision Governance for Enterprise Data Agents starts by accepting that a passing type check proves parseability, and nothing about authorization, currency, semantic consistency, or reversibility.
How can a schema-valid decision still be wrong in production?
A schema-valid decision can be wrong because validation checks form, while production failures come from intent, data, and environment. The benchmarks that measure this gap are unambiguous about its size.
On the BIRD benchmark, built from 95 databases, 33.4 GB of data, 37 professional domains, and 12,751 text-to-SQL pairs, GPT-4's 54.89% execution accuracy means roughly one in two queries returns a well-formed, executable, wrong answer. Human performance on the same tasks is 92.96%. Execution accuracy is the strict measure: the query runs and the result is compared against the correct result set.
The WebArena benchmark shows the same pattern at the task level, and adds a detail that matters for governance. Across 812 long-horizon tasks in four self-hosted web applications, the best GPT-4-based agent reached 14.41% end-to-end task success against 78.24% for humans. Of 61 task templates, GPT-4 reached 100% success on only four. Tasks derived from the same template differ in repetition and complexity, so passing one variation does not predict passing the next.
The "so what" for decision trustworthiness for AI agents: a passing validator tells you the decision is consumable, not that it is right. Treat validation as the entry gate to evaluation, not as the evaluation. Before an agent generates a query at all, the intent and the metric it is answering for need to be fixed, which is the discipline behind resolving intent and metrics before generating SQL.
How do you calibrate confidence and decide when an agent must abstain?
Confidence calibration and abstention for data agents start with one measurement you can run this week: compare stated confidence against observed accuracy on held-out tasks. Expected calibration error is the sample-weighted absolute gap between accuracy and mean confidence across M equally spaced bins; Guo et al. report it with M = 15 bins, and a perfectly calibrated classifier has ECE = 0 (ICML 2017).
Models are not calibrated by default. Guo et al. found that "modern neural networks, unlike those from a decade ago, are poorly calibrated," and that temperature scaling, a single-parameter variant of Platt scaling, is "surprisingly effective at calibrating predictions" (ICML 2017).
Elicitation matters as much as correction. Verbalized confidence cuts calibration error on instruction-tuned models: GPT-3.5-turbo's ECE on TruthfulQA falls from 0.389 with one verbalized guess to 0.203 with four, and GPT-4's from 0.350 to 0.198 (Xiong et al., EMNLP 2023). The same study reports chain-of-thought does not noticeably improve calibration.
The "so what": calibration reduces how often the agent acts wrongly; it does not reduce what happens when it does. Measure ECE on your own task distribution before setting a threshold, because one calibrated on TriviaQA-style recall will not transfer to financial reconciliation. No published abstention-threshold procedure was verified for this guide, so treat threshold selection as your own empirical work rather than a sourced method.
What does permission-aware context require at retrieval and tool-invocation time?
Permission-aware context requires that authorization be enforced at the moment of retrieval and at every tool call, not only at the orchestrator. Microsoft's least-privilege guidance for agents names four controls: a unique, dedicated agent identity with a named owner and approver; default denial of unreviewed tools, plugins, and cross-tenant paths; allowlisting retrieval tools and actions only, with downstream authorization verified on each call; and time-limited privileges through just-in-time entitlements.
AWS's agentic AI lens states the failure mode directly: "Broad permissions turn a misinterpreted instruction into a cascading incident." Its best practice AGENTREL02-BP02 implements this with AgentCore Identity for authentication and AgentCore Policy with Cedar for runtime access boundaries, and rates the risk as High if the practice is not established.
The cross-source finding that ties these together comes from OWASP's LLM Top 10: prompt injection severity "can vary greatly and are largely dependent on both the business context the model operates in, and the agency with which the model is architected." Read against the vendor guidance, permission scoping is a prompt-injection mitigation, not only a governance control.
One limit worth stating: no source connects least privilege to task success. Scoping permissions narrows what a wrong decision can do; it does not make the agent more capable.
Why does semantic consistency determine whether an answer is testable?
Semantic consistency determines testability because two agents can both be correct against their own definitions and still disagree, which makes any comparison meaningless. Unity Catalog metric views address this by separating measure definitions from the dimensions used to group and filter them, so a metric is defined once and queried at runtime. The same documentation recommends adding synonyms, display names, and formatting rules to improve agent accuracy and display consistent formats across tools.
Certification is the trust signal that makes the definition auditable rather than merely shared: Databricks describes certifying a data asset to signal it meets organizational standards, or deprecating it to warn it is outdated (data discovery in Unity Catalog).
The source limit matters here. This documentation covers semantic consistency only. It says nothing about permissions, audit, or agent identity, so it cannot support claims about agent governance. Use it for the definition layer and take the other layers from the sources that address them.
What must a decision record contain to survive an audit?
A decision record must let an auditor reconstruct who acted, under what authority, on what data, with what result. Article 12's record-keeping duty requires high-risk AI systems to "technically allow for the automatic recording of events (logs) over the lifetime of the system," and sets a minimum field set: the period of each use with start and end date/time, the reference database, the input data that produced a match, and identification of the natural persons involved in verifying results.
OpenTelemetry's GenAI agent spans give a concrete field set to build against, including gen_ai.operation.name with values such as create_agent, invoke_agent, plan, and execute_tool, plus gen_ai.agent.name and tool-span attributes. The document is marked Development, so treat it as a working convention rather than a stable standard.
What enterprises actually have in place falls well short. The CSA visibility-gap survey reports on organizations' visibility into agent activity, including real-time registries, session tracing, and audit logging. The survey publishes no sample size or methodology in the available material, so read these as directional.
The CSA visibility-gap survey also finds 19% have none of these controls. JumpCloud's agent-identity survey, fielded July 8 to 20, 2026 with U.S. and U.K. IT decision-makers, reports that 18% require human approval before high-risk agent actions, 46% allow them automatically and log for later review, 38% can fully identify the authorizing identity, and 37% are fully prepared with the records and logs an audit needs.
Why logging everything is not audit-ready
The "so what": logging everything is not the same as being audit-ready. The gap between 46% who allow and log and 37% who are prepared is the difference between having data and having reconstructable audit trails for AI agent decisions.
When must a human be in the loop, and how is that threshold set?
A human must be in the loop when the action is high-impact, hard to reverse, or outside the agent's demonstrated competence, and the threshold should be set per action type rather than per agent. Article 14 of the EU AI Act requires that oversight persons be able to understand the system's capacities and limitations, remain aware of automation bias, correctly interpret output, decide not to use or to override it, and intervene through a stop button or similar procedure that halts the system in a safe state. Article 14(3) requires oversight measures commensurate with the risk, level of autonomy, and context of use.
NIST's generative AI profile names the failure mode this guards against: human-AI configuration risks including automation bias, over-reliance, and algorithmic aversion. A human who rubber-stamps agent output is not a control.
The deployed state does not match the guidance. The CSA visibility-gap survey reports human approval is required for accessing sensitive data by 69% of organizations, for making system changes by 68%, for approving financial transactions by 62%, and for granting permissions by 51%, while 68% rate human-in-the-loop oversight essential or very important. The gap between 68% who call it essential and 51% who gate permission changes is where the escalation policy has to be written.
The "so what": set the threshold by action type, using reversibility and blast radius as the criteria, and test the stop path before you need it.
FAQ
Is a typed decision automatically a trustworthy decision?
No. Typing guarantees the decision is well-formed against a declared contract, not that it is correct, authorized, or reconstructable. The NIST AI Risk Management Framework states that addressing trustworthiness characteristics individually does not ensure trustworthiness, and a type check enforces one property.
How do I measure whether my agent's confidence scores mean anything?
Measure expected calibration error on held-out data from your own task distribution. Expected calibration error is the sample-weighted gap between accuracy and mean confidence across M bins, with M = 15 in the original formulation, and a perfectly calibrated model scores 0. If your ECE is high, verbalized confidence cuts calibration error substantially, with GPT-4's TruthfulQA ECE falling from 0.350 to 0.198 when the model gives four guesses instead of one.
Does least privilege slow agents down?
It adds friction for high-impact operations, and the sources say so. Microsoft's least-privilege guidance for agents lists increased upfront design effort, lifecycle operational complexity, and friction for high-impact operations as trade-offs, without quantifying them. AWS's agentic AI lens rates the risk as High if permissions are not scoped, which is the cost on the other side.
What is the minimum content of an agent decision record?
At minimum, the fields Article 12's record-keeping duty specifies for high-risk systems: the period of each use with start and end date/time, the reference database, the input data that produced a match, and identification of the persons verifying results. OpenTelemetry's GenAI agent spans add the operational fields, including gen_ai.operation.name, gen_ai.agent.name, and tool-span attributes.
How many enterprises are actually audit-ready for agent actions?
Fewer than the adoption numbers suggest. JumpCloud's agent-identity survey reports 37% are fully prepared with the records, logs, and evidence needed for an audit, and 38% can fully identify the identity that authorized an action. The CSA visibility-gap survey reports only 28% can reliably trace agent actions to a human or system across all environments.
Can I skip human review if the agent's confidence is high?
Not on the basis of confidence alone. Article 14 of the EU AI Act requires oversight measures commensurate with the risk and level of autonomy, not with the model's stated confidence, and requires a stop path regardless. NIST's generative AI profile names over-reliance and automation bias as the risks that make confidence-gated review unreliable.
Does passing this checklist prove my agent is safe?
No, and no framework in this space claims it does. The NIST AI Risk Management Framework is voluntary and states that addressing trustworthiness characteristics individually will not ensure trustworthiness. The five layers here are criteria for evaluation, not a certification, and the evidence base for several of them is thinner than the guidance built on top of it.
Conclusion
The five layers separate cleanly, and each one has a testable criterion. Typing guarantees form, not correctness. Validation guarantees the decision is consumable, not right, and the BIRD and WebArena results show how wide that gap runs. Calibration is measurable with ECE and improvable with better confidence elicitation. Permission-aware context is enforceable at retrieval and tool-invocation time, and it doubles as a prompt-injection mitigation. Semantic consistency is what makes two answers comparable at all. Decision records are what make an answer reconstructable, and the survey data shows most organizations are not there yet.
The escalation threshold is the layer that ties them together, because it is where the other four become a policy rather than a set of measurements. Set it by action type, using reversibility and blast radius, and test the stop path before you need it.
Two honest limits. First, the checklist is a set of criteria, not a certification; passing it does not prove safety, and the NIST framework says the same about its own characteristics. Second, the evidence base is uneven. The CSA figures are directional because the survey publishes no methodology, no source connects least privilege to task success, and no published abstention-threshold procedure was verified here. Where the evidence stops, the criterion is still worth testing, but you are setting the threshold on your own data.
The next step is to pick one layer and measure it this quarter. Calibration is usually the fastest to instrument, because ECE needs only held-out predictions and correctness labels you already have.
About the publisher: Singdata is a Singapore-headquartered company that develops data agents on top of its cloud-agnostic, AI-native data infrastructure, and it sells that platform to enterprises. It therefore has a commercial interest in how enterprises govern data agents, and readers should weigh its perspective accordingly. The guidance above does not depend on Singdata's products.