8 questions from the RFI stage, free
These come from the module as sold. The workbook adds follow-ups, a response format, a weight and a score column to each.
1. Describe how customers configure the set of actions an agent is permitted to take, including whether permissions can be scoped per action type rather than granted as a single bundle.
Why it matters. All-or-nothing permission models force customers to over-grant authority to agents, expanding blast radius from any single failure or prompt injection. OWASP LLM06:2025 Excessive Agency names excessive permissions as a root cause and recommends limiting permissions to the minimum necessary. This question screens vendors whose permission model is too coarse for regulated environments.
- Describes named permission primitives at per-action or per-tool granularity.
- References a policy or role configuration surface (UI, API, or IaC).
- Distinguishes read-only actions from state-changing actions.
- Only a single 'agent enabled' toggle exists.
- Permissions inherit entirely from the user session with no agent-specific scoping.
- Permission model described only in marketing language without enumerable scopes.
2. Provide a table of external tools, APIs, or systems your agents can invoke out of the box, indicating which are enabled by default.
Why it matters. The tool catalog defines the agent's reach into customer systems and the public internet. Defaults apply to every tool the customer has not reviewed. This question surfaces the baseline attack surface.
- Provides a clear, categorized list of tools.
- Defaults are conservative (most tools are disabled until explicitly enabled).
- Distinguishes between vendor-provided and customer-configured tools.
- Vague references to 'many integrations' without a list.
- Web browsing or arbitrary code execution is enabled by default.
- No way to disable individual tools.
3. Describe the isolation boundary between agent runs, including whether each run executes in a separate process, container, or VM, and what state, if any, can persist between runs.
Why it matters. Cross-run contamination — memory, files, credentials, or context bleeding from one customer's run to another — is a tenancy-breaking failure. Buyers need to understand the concrete isolation primitives, not just marketing assurances.
- Names a specific isolation technology (e.g., gVisor, Firecracker, Kata, ephemeral containers).
- Guarantees an ephemeral filesystem and memory per run.
- Confirms no shared writable state between runs.
- Isolation is described only as 'logical' or 'secure by design'.
- The same process serves multiple agent runs.
- A shared writable filesystem exists across runs.
4. Describe the kill-switch controls available to a customer administrator to halt all agent activity within their tenant immediately.
Why it matters. When an agent is misbehaving — looping, exfiltrating data, or executing erroneous actions — customers need a single control to stop everything without waiting for vendor support. Absence of a tenant-level kill-switch is a serious operational gap.
- A single-action, tenant-wide pause is available to customer administrators.
- The vendor states a documented time-to-effect for the pause.
- Distinct controls exist for blocking new runs versus stopping in-flight runs.
- Pausing requires a vendor support ticket.
- No way to stop in-flight runs.
- The pause takes effect only at a next scheduled boundary (e.g., end of hour).
5. Explain whether an agent's permissions can be scoped per agent run or per task, and how those scopes are enforced at execution time.
Why it matters. Long-lived agent identities that retain a superset of permissions across runs are a major risk. Per-run or per-task scoping limits exposure when an individual run is compromised or misbehaves. The enforcement mechanism matters as much as the configuration surface.
- Describes ephemeral scoped credentials or tokens per run.
- Names the enforcement point (e.g., gateway, policy engine, runtime).
- Scopes are derived from task definition or invoking user, not a static agent identity.
- Agent uses a single long-lived service principal.
- Scoping described as 'best effort' or convention only.
- No runtime enforcement, only pre-prompt instructions.
6. Describe how customers authorize the agent to act on external systems (e.g., OAuth, service accounts, scoped tokens) and what permission scopes are requested by default.
Why it matters. Over-scoped credentials let a hijacked or malfunctioning agent act beyond its task; OWASP LLM06:2025 Excessive Agency names excessive permissions as a root cause. Buyers need to understand the credential model and whether least-privilege is the vendor default or a customer responsibility.
- Supports OAuth with granular scopes.
- Documents the minimum scope required per integration.
- Supports customer-managed service accounts and bring-your-own-credentials (BYOC).
- Requires full-access or admin-level tokens for common integrations.
- No documentation of required scopes.
- Credentials stored in plaintext or passed directly in LLM context.
7. Describe the isolation boundary between the agent's execution environment and your production control plane, including how the agent is prevented from affecting other tenants or core vendor infrastructure.
Why it matters. An agent that can pivot from its sandbox into vendor infrastructure or other tenants is a critical supply-chain risk. The buyer needs to understand both the technical boundary and the threat model the vendor designed against.
- Names network and identity boundaries between the agent runtime and control plane.
- Confirms no agent-runtime access to vendor secrets or shared databases.
- Tenant-level isolation is enforced by infrastructure, not just application logic.
- Agent runtime shares credentials with the control plane.
- Tenant isolation is enforced only by application-layer filtering.
- No penetration testing of agent escape scenarios has been conducted.
8. Describe automated guardrails that halt or quarantine an agent run when anomalous behavior is detected (e.g., loops, runaway spend, repeated errors, policy violations).
Why it matters. Humans cannot watch every agent run in real time. Automated circuit breakers can halt a run when no one is watching. Buyers need to know what conditions trigger automatic intervention and what the default behavior is.
- Names specific trigger conditions (e.g., loop detection, tool-call thresholds, error rate).
- Default action is to halt the run and notify an administrator.
- Customer can tune the thresholds for these triggers.
- No automated halts; the system relies entirely on hard caps.
- Vendor cannot describe what 'anomalous' means in their system.
- No notification is sent when an automatic halt occurs.
What the audit changed
A language model drafted these questions and a second model critiqued them. Three audit passes followed and made 92 changes. Three examples:
Wrong or outdated citation
Draft: Are tool descriptions cryptographically signed by the server?
Now: How does the platform detect a change to a tool's description or schema after an administrator approved the server?
The MCP specification defines no signature for tool descriptions, so the follow-up asks about a mechanism the protocol does not have (pass 1 made the same fix in deep-dive.009). Source: MCP specification, revision 2026-07-28, Tools (https://modelcontextprotocol.io/specification/2026-07-28/server/tools).
Wrong or outdated citation
Draft: Buyer-hosted tool servers supported with mTLS or signed tokens
Now: Buyer-hosted MCP servers supported using the MCP authorization specification (OAuth 2.1 with resource indicators per RFC 8707 and protected resource metadata per RFC 9728), or mTLS
MCP authorization is optional. Where an HTTP-based implementation supports it, the specification is based on OAuth 2.1 (draft-ietf-oauth-v2-1-13): MCP clients MUST implement RFC 8707 resource indicators and MCP servers MUST implement RFC 9728 protected resource metadata, and servers MUST NOT accept or pass through tokens not issued for them. It defines no mTLS or signed-token mechanism; mTLS stays in the signal as a transport option outside MCP. Source: MCP specification, revision 2026-07-28, Authorization (https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization). (Source corrected in pass 2.)
Wrong or outdated citation
Draft: Tool descriptor validation against schema and signature
Now: Tool definitions validated against their declared schema, pinned after admin review, and re-reviewed when a server changes them
The MCP specification defines no signature for tool definitions, so a vendor cannot validate one; it requires clients to treat tool annotations as untrusted unless they come from trusted servers. Source: MCP specification, revision 2026-07-28, Tools (https://modelcontextprotocol.io/specification/2026-07-28/server/tools). (Source corrected in pass 2.)
