How to Test Agents That Will Lie to You, Leak Your Data, and Fail in Production

The Grok CLI incident crystallizes something that adversarial testing frameworks need to make explicit: the model layer and the tool execution layer have different trust surfaces, and testing only one gives you false confidence. A model can be perfectly aligned and still invoke a tool that causes harm — because the tool itself has no awareness of the agent's intended scope. Your egress boundary test needs to fire at the tool execution layer, asserting that file-read and network-write calls are scoped to a declared workspace manifest before they execute, not after. This is a precondition check, not a post-hoc audit. Source: [The Pulse: Grok's CLI caught uploading all your local files to the cloud] — Pragmatic Engineer (https://newsletter.pragmaticengineer.com/p/the-pulse-groks-cli-caught-uploading)

From a platform engineering standpoint, the practical fix is an outbound tool call interceptor in your agent harness that compares each tool invocation against a session-scoped capability manifest. If the manifest says the agent has read access to `/workspace/project` and the tool call references `/home/user/.ssh`, that's a test failure — and in production, an automatic session termination with a logged incident. [AI Synthesis] This pattern maps directly onto the guardrail-as-async-filter trend, but applied to tool execution rather than LLM output: the filter sits between the agent's intent and the tool's action.

// TypeScript: Tool call interceptor with workspace scope enforcement
const WORKSPACE_MANIFEST: WorkspaceScope = {
  allowedReadPaths: ['/workspace/project'],
  allowedWritePaths: ['/workspace/project/output'],
  allowedNetworkHosts: ['api.internal.company.com'],
};

async function interceptToolCall(
  toolName: string,
  toolArgs: Record<string, unknown>,
  sessionId: string,
  anomalyAccumulator: AnomalyAccumulator
): Promise<ToolCallDecision> {
  if (toolName === 'read_file') {
    const targetPath = toolArgs['path'] as string;
    const isAllowed = WORKSPACE_MANIFEST.allowedReadPaths.some(
      (allowed) => targetPath.startsWith(allowed)
    );
    if (!isAllowed) {
      await anomalyAccumulator.record(sessionId, {
        type: 'EGRESS_BOUNDARY_VIOLATION',
        toolName,
        violatingArg: targetPath,
        severity: 'HIGH',
        timestamp: new Date().toISOString(),
      });
      const score = await anomalyAccumulator.getScore(sessionId);
      if (score >= TERMINATION_THRESHOLD) {
        return { action: 'TERMINATE', reason: 'Repeated egress boundary violations' };
      }
      return { action: 'BLOCK', reason: `Path ${targetPath} outside workspace scope` };
    }
  }
  return { action: 'ALLOW' };
}

The CrewAI framing around building agents where data already lives identifies the real production gap: it's not orchestration complexity, it's the absence of data-layer validation. Most agent testing focuses on prompt-response pairs, but when the agent has a live database tool, the query it constructs is the behavior that needs asserting. A semantic assertion engine for SQL-generating agents would validate that every generated query respects the row-level security model of the target database — not by running it and checking the result, but by analyzing the query's table and column references against a declared permission graph before execution. Source: [How to build Agents Where Data Already Lives] — CrewAI Blog (https://blog.crewai.com/how-to-build-agents-where-data-already-lives/)

AIThis connects directly to the broader pattern of the enterprise AI tooling gap: the tooling to build data-connected agents has matured significantly faster than the tooling to validate what those agents do with data access. Platform teams shipping these agents in production need a pre-execution query analyzer sitting in the tool wrapper — one that compares the constructed query against a schema manifest and a permission graph, fails the test if there's a mismatch, and logs the full query for audit. That's not a feature of any major agent framework today; it's something platform teams are building themselves.

// TypeScript: Pre-execution SQL query validator for agent tool wrapper
interface QueryValidationResult {
  allowed: boolean;
  violations: string[];
  sanitizedQuery?: string;
}

async function validateAgentQuery(
  rawQuery: string,
  principalId: string,
  schemaManifest: SchemaManifest,
  permissionGraph: PermissionGraph
): Promise<QueryValidationResult> {
  const parsed = parseSQL(rawQuery); // e.g., node-sql-parser
  const referencedTables = extractTableReferences(parsed);
  const referencedColumns = extractColumnReferences(parsed);
  const violations: string[] = [];

  for (const table of referencedTables) {
    if (!permissionGraph.canRead(principalId, table)) {
      violations.push(`Unauthorized table access: ${table}`);
    }
    const currentColumns = schemaManifest.getColumns(table);
    for (const col of referencedColumns.filter(c => c.table === table)) {
      if (!currentColumns.includes(col.name)) {
        violations.push(`Schema drift detected: column ${col.name} not in ${table}`);
      }
    }
  }

  if (violations.length > 0) {
    return { allowed: false, violations };
  }

  const sanitized = injectRLSFilters(rawQuery, principalId, permissionGraph);
  return { allowed: true, violations: [], sanitizedQuery: sanitized };
}

The Towards Data Science framing on AI-native enterprise data platforms identifies an architectural pattern that directly addresses the production shipping gap: embedding AI-powered QA as a pipeline stage rather than a post-deployment monitoring afterthought. In practice, this means a supervisor-style node in your LangGraph or CrewAI pipeline that holds a policy spec — expressed as a set of semantic assertions — and evaluates each agent output against it before passing state forward. The assertions can range from structural (output matches expected schema) to semantic (output does not reference PII fields the requesting principal lacks access to) to behavioral (agent did not invoke a tool outside its declared capability set in this session). Source: [Many Companies Use AI. Few Know How to Build an AI-Native Enterprise Data Platform.] — Towards Data Science (https://towardsdatascience.com/many-companies-use-ai-few-know-how-to-build-an-ai-native-enterprise-data-platform/)

AIThe missing piece in most platform teams' governance stories is that the QA agent needs its own test suite — adversarial cases designed to verify that the QA agent itself cannot be circumvented by a well-crafted agent output. This is the recursive testing problem: the guardrail needs its own red team. That's not paranoia; it's the same discipline you'd apply to any security-critical component. Run adversarial outputs through the QA agent in CI and assert that it rejects them — if it passes a policy-violating output, that's a test failure in the governance layer, not just a runtime incident.

// TypeScript: Inline QA agent node for LangGraph pipeline
interface PolicySpec {
  schemaAssertions: SchemaAssertion[];
  semanticAssertions: SemanticAssertion[];
  capabilityAuditRules: CapabilityRule[];
}

async function qaAgentNode(
  state: AgentState,
  policySpec: PolicySpec,
  llmClient: LLMClient
): Promise<Partial<AgentState>> {
  const output = state.lastAgentOutput;
  const violations: PolicyViolation[] = [];

  // Schema assertions — fast, deterministic
  for (const assertion of policySpec.schemaAssertions) {
    const result = assertion.evaluate(output);
    if (!result.passes) violations.push({ type: 'SCHEMA', ...result });
  }

  // Semantic assertions — LLM-evaluated against policy spec
  if (violations.length === 0) {
    const semanticResult = await llmClient.evaluate({
      systemPrompt: buildSemanticAssertionPrompt(policySpec.semanticAssertions),
      userContent: JSON.stringify(output),
      responseSchema: SemanticAssertionResultSchema,
    });
    violations.push(...semanticResult.violations);
  }

  // Capability audit — check tool call log against session manifest
  const capabilityViolations = auditToolCallLog(
    state.toolCallLog,
    state.sessionCapabilityManifest,
    policySpec.capabilityAuditRules
  );
  violations.push(...capabilityViolations);

  const highSeverity = violations.filter(v => v.severity === 'HIGH');
  if (highSeverity.length > 0) {
    return { blocked: true, violations, nextNode: 'terminate' };
  }

  const mediumSeverity = violations.filter(v => v.severity === 'MEDIUM');
  if (mediumSeverity.length > 0) {
    return { pendingHumanReview: true, violations, nextNode: 'human_review' };
  }

  return { qaApproved: true, nextNode: 'output' };
}

The Archaeologist's Copilot piece on AI-assisted Java 1.5 modernization surfaces a validation discipline that translates directly into agent testing: the breakthrough came not from better prompts but from grounding every AI suggestion in evidence — running it in a stable, isolated environment, asserting against observable behavior, not LLM confidence. The same structure applies to testing data-access agents. If your agent claims to have retrieved and summarized a customer record, your test harness should independently fetch that record from the test database and run a structured diff against the agent's output, asserting that key fields are accurately represented and no hallucinated values are present. Source: [The Archaeologist's Copilot] — Martin Fowler (https://martinfowler.com/articles/archaeologist-copilot.html)

AIThis connects back to the data-boundary testing pattern from block_2: the semantic assertion engine for agent outputs needs a ground-truth retrieval step built into it. You're not evaluating whether the output sounds right; you're evaluating whether it accurately reflects a specific, retrievable artifact. That's a fundamentally different test design, and it's the difference between a test that catches hallucinations and a test that only catches structural malformation. For platform teams running agents in high-stakes production environments — financial data, health records, legal documents — this isn't optional rigor; it's the minimum bar.

// TypeScript: Evidence-grounded assertion for agent output validation
interface GroundTruthAssertion {
  field: string;
  extractFromOutput: (output: AgentOutput) => unknown;
  fetchGroundTruth: (db: DatabaseClient, context: TestContext) => Promise<unknown>;
  comparator: (agentValue: unknown, groundTruth: unknown) => AssertionResult;
}

async function runEvidenceGroundedTest(
  agent: Agent,
  testInput: AgentInput,
  assertions: GroundTruthAssertion[],
  db: DatabaseClient,
  context: TestContext
): Promise<TestResult> {
  const output = await agent.run(testInput);
  const fieldViolations: FieldViolation[] = [];

  for (const assertion of assertions) {
    const agentValue = assertion.extractFromOutput(output);
    const groundTruth = await assertion.fetchGroundTruth(db, context);
    const result = assertion.comparator(agentValue, groundTruth);

    if (!result.passes) {
      fieldViolations.push({
        field: assertion.field,
        agentValue,
        groundTruth,
        delta: result.delta,
        // Evidence trail: what the agent actually queried
        queryLog: output.toolCallLog.filter(t => t.tool === 'query_database'),
      });
    }
  }

  return {
    passed: fieldViolations.length === 0,
    violations: fieldViolations,
    evidenceTrail: output.toolCallLog,
  };
}