Critic Agents as Sentinels: Designing Feedback Loops That Catch Cascading Failures Before They Reach Users

The cybersecurity eval pattern described at eugeneyan.com structures evaluation around four components: a sandboxed target, difficulty-graded inputs, agent-available tools, and a grader that tracks progress along the attack chain. The insight worth stealing for critic agent design is the attack chain grader — it doesn't wait for a full exploit to declare failure. It scores how far along a bad trajectory the agent has traveled, which gives you a leading indicator rather than a lagging one. For a platform critic monitoring a multi-agent pipeline, this translates to scoring intermediate outputs against a failure taxonomy: does this tool call look like the first step in a context poisoning sequence? Does this subagent output pattern match the early signature of a hallucination cascade? The grader becomes a continuous risk signal, not a checkpoint gate. Source: [Patterns for Building Cybersecurity Evals] — eugeneyan.com (https://eugeneyan.com//writing/cybersecurity-evals/)

AIConnecting this to the broader governance maturation trend: the move from binary output validation to trajectory-aware scoring is architecturally significant. A critic that only checks final outputs is like a circuit breaker that trips after the fire has already started. A trajectory-aware critic that scores intermediate states is closer to a smoke detector — it catches the precondition, not the consequence. For Sam's TypeScript async processors, this means your critic doesn't just validate what a subagent returned; it maintains a rolling risk score across the last N steps in the agent's execution trace and triggers on score acceleration, not score threshold breach.

// TypeScript: Trajectory-aware critic with rolling risk accumulation
type StepRiskScore = { stepId: string; score: number; timestamp: number };

const RISK_ACCELERATION_WINDOW = 5;
const ACCELERATION_THRESHOLD = 0.15; // score delta per step

function detectRiskAcceleration(history: StepRiskScore[]): boolean {
  if (history.length < RISK_ACCELERATION_WINDOW) return false;
  const window = history.slice(-RISK_ACCELERATION_WINDOW);
  const deltas = window.slice(1).map((s, i) => s.score - window[i].score);
  const avgDelta = deltas.reduce((a, b) => a + b, 0) / deltas.length;
  return avgDelta > ACCELERATION_THRESHOLD;
}

async function criticEvaluateStep(
  step: AgentStep,
  history: StepRiskScore[],
  supervisor: SupervisorNode
): Promise<void> {
  const score = await scoreAgainstFailureTaxonomy(step);
  history.push({ stepId: step.id, score, timestamp: Date.now() });

  if (detectRiskAcceleration(history)) {
    await supervisor.escalate({ reason: 'risk_acceleration', step, history });
    return;
  }
  if (score > 0.85) {
    await supervisor.escalate({ reason: 'critical_score', step, history });
  } else if (score > 0.6) {
    await supervisor.reroute(step);
  }
}

The brownfield agentic engineering challenge is that legacy systems carry constraints that aren't encoded anywhere an agent can read — they live in Confluence pages, in the heads of engineers who left three years ago, or in the specific order that database migrations ran in 2019. The zone classification approach forces that hidden risk surface into a structured artifact that both humans and critic agents can reason over. A red-zone designation isn't just a warning label; it's a governance signal that should propagate through your critic's scoring logic. When a subagent's action touches a red-zone component, the critic's failure taxonomy should weight conservatively — assume brittleness, assume undocumented dependencies, assume that an apparently valid output can still produce a bad downstream effect. Source: [Brownfield Agentic Engineering] — addyosmani.com (https://addyosmani.com/blog/brownfield-agentic-engineering/)

AIThis connects directly to pillar-based safety design: zone classification is a static pillar, evaluated once (or on a slow refresh cycle), that constrains agent autonomy before the critic ever fires at runtime. Think of it as a pre-filter — the critic doesn't need to reason from scratch about whether an action is risky if the zone map already tells it the execution environment is inherently fragile. In Sam's PostgreSQL-backed platform context, this means storing zone metadata in a governance table that agent runtimes query at startup, so the risk profile of a given database schema, API endpoint, or service boundary is a first-class input to critic scoring — not something inferred dynamically from agent behavior alone.

// TypeScript: Zone-aware critic threshold configuration
type ZoneClassification = 'green' | 'yellow' | 'red';

interface ZoneConfig {
  classification: ZoneClassification;
  escalationThreshold: number;
  requireHumanOnEscalation: boolean;
  knownFragilities: string[];
}

const ZONE_THRESHOLDS: Record<ZoneClassification, Pick<ZoneConfig, 'escalationThreshold' | 'requireHumanOnEscalation'>> = {
  green:  { escalationThreshold: 0.80, requireHumanOnEscalation: false },
  yellow: { escalationThreshold: 0.55, requireHumanOnEscalation: false },
  red:    { escalationThreshold: 0.30, requireHumanOnEscalation: true },
};

async function getZoneAwareCriticConfig(
  componentId: string,
  db: PostgresClient
): Promise<ZoneConfig> {
  const row = await db.query<ZoneConfig>(
    `SELECT classification, escalation_threshold, require_human_on_escalation, known_fragilities
     FROM governance.component_zones WHERE component_id = $1`,
    [componentId]
  );
  return {
    ...row,
    ...ZONE_THRESHOLDS[row.classification],
  };
}

The October 14th agentic engineering BoF in SF is framed around experimental work people haven't discussed publicly — edge cases, weird behaviors, things that haven't been figured out yet. That framing is directly relevant to governance: the failure modes that automated critics miss are almost always the ones that haven't been formally observed enough to encode into a scoring taxonomy. Practitioner sharing sessions create a feedback channel that runs parallel to your automated critic stack, and the signal quality is high precisely because it's unfiltered. An engineer describing a weird tool-call loop they observed last Tuesday is more useful for critic taxonomy development than a post-incident report written two weeks later after the edges have been sanded off. Source: [SF October 14th: A Birds of a Feather Session on Agentic Engineering] — simonwillison.net (https://simonwillison.net/2026/Sep/23/bof-agentic-engineering/)

AIConnecting this to the brownfield zone model from block_2: red-zone fragilities often live exclusively in practitioner memory, not in documentation or test coverage. A structured sharing session is one of the few mechanisms that can extract that tacit knowledge and turn it into something a critic can act on. For Sam's platform team, this suggests a lightweight monthly practice: a 45-minute async-first retrospective where engineers submit agentic anomalies they observed — no post-incident formality required — and the platform team triages which ones should become new critic scoring rules. The human loop closes back into the automated one.

// TypeScript: Anomaly registry that feeds critic taxonomy updates
interface AnomalyReport {
  reportedBy: string;
  observedAt: Date;
  agentId: string;
  description: string;
  stepTrace: AgentStep[];
  zoneClassification?: ZoneClassification;
  recurrenceCount: number;
}

async function submitAnomalyReport(
  report: Omit<AnomalyReport, 'recurrenceCount'>,
  db: PostgresClient
): Promise<{ shouldEscalateToTaxonomy: boolean }> {
  const existing = await db.query<{ id: string; recurrenceCount: number }>(
    `SELECT id, recurrence_count FROM governance.anomaly_registry
     WHERE description_hash = hash_description($1)`,
    [report.description]
  );

  if (existing) {
    const updated = await db.query<AnomalyReport>(
      `UPDATE governance.anomaly_registry
       SET recurrence_count = recurrence_count + 1, last_seen = NOW()
       WHERE id = $1 RETURNING *`,
      [existing.id]
    );
    return { shouldEscalateToTaxonomy: updated.recurrenceCount >= 3 };
  }

  await db.query(
    `INSERT INTO governance.anomaly_registry (reported_by, agent_id, description, step_trace, zone)
     VALUES ($1, $2, $3, $4, $5)`,
    [report.reportedBy, report.agentId, report.description,
     JSON.stringify(report.stepTrace), report.zoneClassification]
  );
  return { shouldEscalateToTaxonomy: false };
}

The Copilot runtime migration from TypeScript/Node.js/V8 to Rust is notable not just as an engineering achievement but as a governance case study. The team used the Copilot app and CLI to execute the rewrite — a self-referential loop where the agent being rewritten is also doing the rewriting. The critic governance challenge here is subtle: your failure taxonomy for the old runtime is now partially invalid for the new one. Memory management patterns, async behavior, error propagation semantics — these all change in ways that require the critic's scoring model to be updated in lockstep with the migration. A critic that keeps flagging Rust borrow-checker patterns as anomalies because it learned TypeScript norms will generate enough noise to erode trust in the feedback signal. Source: [Migrating the GitHub Copilot runtime to Rust, using Copilot] — github.blog (https://github.blog/ai-and-ml/generative-ai/migrating-the-github-copilot-runtime-to-rust-using-copilot/)

AIThis connects to the brownfield zone model in an interesting way: a runtime undergoing active migration should temporarily be classified as red-zone regardless of its normal designation. The zone map isn't static; it should be driven by a change-rate signal. A component that normally sits in yellow becomes red during a migration window — and the critic's thresholds should tighten automatically in response. For Sam's platform, this means wiring your deployment pipeline to push zone reclassification events to the governance store whenever a major runtime or infrastructure change is initiated, so the critic stack adapts before the change lands rather than after.

// TypeScript: Migration-aware zone reclassification event handler
interface MigrationEvent {
  componentId: string;
  migrationType: 'runtime_rewrite' | 'schema_migration' | 'dependency_upgrade';
  previousRuntime?: string;
  targetRuntime?: string;
  initiatedBy: string;
  estimatedCompletionDate: Date;
}

async function handleMigrationStart(
  event: MigrationEvent,
  db: PostgresClient,
  criticConfigBus: EventEmitter
): Promise<void> {
  // Temporarily reclassify to RED during migration window
  await db.query(
    `UPDATE governance.component_zones
     SET classification = 'red',
         reclassification_reason = $1,
         reclassification_expires = $2,
         previous_classification = classification
     WHERE component_id = $3`,
    [
      `Migration in progress: ${event.migrationType} to ${event.targetRuntime}`,
      event.estimatedCompletionDate,
      event.componentId
    ]
  );

  // Push config update to live critic agents
  criticConfigBus.emit('zone:reclassified', {
    componentId: event.componentId,
    newClassification: 'red',
    reason: event.migrationType,
    invalidateBaseline: event.migrationType === 'runtime_rewrite',
  });
}

High-throughput AI pipeline architectures like GPT-4o Live are built around aggressive latency optimization — the engineering concerns center on token streaming, session management, and minimizing round-trip overhead. The implicit lesson for critic design is that any synchronous critic that adds meaningful latency to the hot path will either get disabled under load or create backpressure that degrades the user experience it's supposed to protect. The pattern that survives production is a two-tier critic: a synchronous heuristic layer that runs in microseconds and handles the clear-cut cases (format violations, known bad patterns, zone boundary crossings), and an asynchronous inference-backed critic that evaluates the ambiguous cases against the full failure taxonomy and writes results back to a shared risk store that future steps consult. Source: [How OpenAI Built GPT-Live] — blog.bytebytego.com (https://blog.bytebytego.com/p/how-openai-built-gpt-live)

AIThis two-tier architecture maps cleanly onto the attack-chain grading pattern from block_1: the heuristic pre-filter handles step-level pattern matching (fast, cheap, deterministic), while the inference-backed critic does trajectory-level scoring (slower, probabilistic, triggered selectively). The zone classification from block_2 acts as a routing signal between tiers — green-zone steps might only ever hit the heuristic tier, while red-zone steps always escalate to full inference evaluation. The result is a critic stack that's budget-aware by construction, not by accident.

// TypeScript: Two-tier critic with async inference and shared risk store
interface CriticTier1Result {
  blocked: boolean;
  reason?: string;
  shouldEscalateToTier2: boolean;
}

function tier1HeuristicCritic(step: AgentStep, zone: ZoneClassification): CriticTier1Result {
  const hasKnownBadPattern = KNOWN_BAD_PATTERNS.some(p => p.test(step.output));
  if (hasKnownBadPattern) return { blocked: true, reason: 'known_bad_pattern', shouldEscalateToTier2: false };
  const shouldEscalateToTier2 = zone === 'red' || step.toolCallCount > 3 || step.confidence < 0.7;
  return { blocked: false, shouldEscalateToTier2 };
}

async function tier2InferenceCritic(
  step: AgentStep,
  riskStore: Redis,
  taxonomy: FailureTaxonomy
): Promise<void> {
  const score = await scoreAgainstTaxonomy(step, taxonomy);
  const sessionKey = `risk:${step.sessionId}`;
  const current = parseFloat(await riskStore.get(sessionKey) ?? '0');
  // Exponential moving average — recent steps weighted higher
  const updated = 0.7 * score + 0.3 * current;
  await riskStore.set(sessionKey, updated.toString(), 'EX', 3600);
  if (updated > 0.75) {
    await supervisor.escalate({ step, riskScore: updated, source: 'tier2_inference' });
  }
}