Per-Phase Provenance Logging: Correlating Agent Execution Traces with Real-Time Governance Signals

The practical design here is straightforward: when your orchestrator transitions between phases — say, from retrieval to reasoning to tool execution — emit a phase-boundary span with a stable correlation ID, then use that same ID as the requesting entity context when fetching credentials from Vault. Vault's audit log will record the policy evaluation against that identity, and your tracing backend can join on the correlation ID to reconstruct the full picture: what the agent was doing, what it was allowed to do, and what it actually fetched. This is the difference between an audit log that tells you a credential was accessed at 14:32 and one that tells you the summarization phase of workflow W-4471 fetched the database read token under the restricted-analyst policy. Source: [HashiCorp Vault agentic IAM is now generally available] — HashiCorp (https://www.hashicorp.com/blog/hashicorp-vault-agentic-iam-is-now-generally-available)

AIWhere this connects to per-phase provenance more broadly: IAM events are often siloed from execution traces because they live in different systems — Vault audit logs in one place, OpenTelemetry spans in another. The engineering discipline is to treat the phase correlation ID as the join key across both, emitted at phase entry and propagated as a baggage header or request context through every downstream call the phase makes.

// TypeScript: emit phase boundary span and propagate correlation ID into Vault fetch
import { trace, context, propagation } from '@opentelemetry/api';

async function executePhase(
  phaseName: string,
  correlationId: string,
  fn: () => Promise<void>
): Promise<void> {
  const tracer = trace.getTracer('agent-orchestrator');
  const span = tracer.startSpan(`phase.${phaseName}`, {
    attributes: { 'agent.phase': phaseName, 'workflow.correlationId': correlationId }
  });
  const ctx = trace.setSpan(context.active(), span);

  await context.with(ctx, async () => {
    // Propagate W3C trace context — Vault client reads this as request metadata
    const headers: Record<string, string> = {};
    propagation.inject(context.active(), headers);

    await vaultClient.getSecret('db-read-token', {
      entityAlias: `agent-${phaseName}`,
      requestMetadata: { correlationId, ...headers }
    });

    await fn();
  });

  span.end();
}

The insight from constraint-based code quality work is that agents produce code at a rate that makes manual review structurally impossible, so the gates become the review. But a gate that just blocks a PR and logs to a CI system is a dead end for observability — you know something failed, but not which agent invocation, which phase, or what the agent's reasoning state was when it generated the problematic code. The fix is to emit gate evaluation results as structured spans back into your tracing pipeline, carrying the phase correlation ID, the specific constraint that failed, the metric value, and the threshold. Now you can ask: 'Show me all reasoning phases where the agent's code cleared unit tests but failed mutation testing' — and that's a genuinely diagnostic query. Source: [Agentic Code Quality] — Addy Osmani (https://addyosmani.com/blog/agentic-code-quality/)

AIThis pattern extends naturally to non-code agents: any phase that produces an artifact — a plan, a tool call sequence, a summarization — can be evaluated against constraints (schema validation, confidence thresholds, semantic similarity checks) and the evaluation result emitted as a typed provenance event. The gate becomes the observable boundary between phases, and the structured result is what makes the provenance log queryable rather than just archival.

// TypeScript: emit constraint gate results as structured provenance spans
interface GateResult {
  gateName: string;
  passed: boolean;
  metricValue?: number;
  threshold?: number;
  correlationId: string;
  phaseId: string;
}

async function runGateWithProvenance(
  gateFn: () => Promise<{ value: number; threshold: number }>,
  gateName: string,
  correlationId: string,
  phaseId: string
): Promise<GateResult> {
  const tracer = trace.getTracer('quality-gates');
  const span = tracer.startSpan(`gate.${gateName}`, {
    attributes: {
      'gate.name': gateName,
      'workflow.correlationId': correlationId,
      'agent.phaseId': phaseId
    }
  });

  try {
    const { value, threshold } = await gateFn();
    const passed = value >= threshold;
    span.setAttributes({
      'gate.passed': passed,
      'gate.metricValue': value,
      'gate.threshold': threshold
    });
    return { gateName, passed, metricValue: value, threshold, correlationId, phaseId };
  } finally {
    span.end();
  }
}

ToolGrad's core contribution is inverting the data generation loop: instead of generating queries and hoping the agent finds a valid tool path, generate the tool path first and derive the query from it. The production implication is that your execution logs already contain exactly these answer-first records — every completed tool-call chain is a solved example with real inputs, real tool responses, and a real outcome. But only if your logging architecture captures the full chain with phase attribution, not just the terminal output. The schema question is: for each tool invocation, are you recording the phase it belongs to, the agent that issued it, the input arguments, the raw response, and the correlation ID that links it back to the workflow span? Source: [ToolGrad: Efficient tool-use dataset generation with textual gradients] — Google Research (https://research.google/blog/toolgrad-efficient-tool-use-dataset-generation-with-textual-gradients/)

AIThis connects directly to the constraint gate pattern in block_2: a tool call that succeeds mechanically but produces an output that fails a downstream quality gate is exactly the kind of labeled negative example that makes fine-tuning useful. If your provenance log captures both the tool invocation record and the gate evaluation result under the same correlation ID, you have a structured training example without any manual annotation — the gate result is the label.

// TypeScript: wrap tool calls to emit phase-attributed provenance records
type ToolInvocationLog = {
  correlationId: string;
  phaseId: string;
  phaseName: string;
  toolId: string;
  inputArgs: unknown;
  response: unknown;
  durationMs: number;
  timestamp: string;
};

async function invokeToolWithProvenance<TArgs, TResponse>(
  tool: (args: TArgs) => Promise<TResponse>,
  toolId: string,
  args: TArgs,
  phase: { id: string; name: string },
  correlationId: string,
  emit: (log: ToolInvocationLog) => void
): Promise<TResponse> {
  const start = Date.now();
  const response = await tool(args);
  emit({
    correlationId,
    phaseId: phase.id,
    phaseName: phase.name,
    toolId,
    inputArgs: args,
    response,
    durationMs: Date.now() - start,
    timestamp: new Date().toISOString()
  });
  return response;
}

The healthcare MLR analysis case illustrates a general pattern: complex multi-factor variance analysis is a long-horizon reasoning task that unfolds across several retrieval and computation phases, and the trustworthiness of the final answer depends entirely on whether each phase drew from the right data with the right context. The engineering discipline is to treat each decomposition step as a named phase with its own provenance record — data source fetched, confidence score or freshness metadata, intermediate result, and the reasoning step that consumed it. Without that, you get an answer you cannot validate, which is fine for a demo and a liability in production. Source: [Health Plans: Your BI Tells You MLR Moved. Can Your AI Tell You Why?] — Databricks (https://www.databricks.com/blog/health-plans-your-bi-tells-you-mlr-moved-can-your-ai-tell-you-why)

AIThis is the same structural problem as the context fragmentation issue flagged in the longitudinal trends: when an agent's reasoning depends on retrieved context that is not explicitly logged per phase, failures become opaque. The Vault IAM pattern from block_1 and the gate evaluation pattern from block_2 both apply here — log what data was accessed, under which policy, with what freshness, and whether downstream constraints on the result passed or failed. The provenance record is the same regardless of whether the agent is writing code or decomposing a financial variance.

// TypeScript: log each decomposition step as a named phase provenance record
interface DecompositionPhaseLog {
  correlationId: string;
  phaseIndex: number;
  phaseName: string;
  dataSources: Array<{ name: string; freshnessMs: number; validatedAt: string }>;
  intermediateResult: unknown;
  confidenceScore?: number;
  timestamp: string;
}

async function runDecompositionPhase(
  phaseName: string,
  phaseIndex: number,
  correlationId: string,
  fetchData: () => Promise<{ sources: DecompositionPhaseLog['dataSources']; result: unknown; confidence?: number }>,
  emit: (log: DecompositionPhaseLog) => void
): Promise<unknown> {
  const { sources, result, confidence } = await fetchData();
  emit({
    correlationId,
    phaseIndex,
    phaseName,
    dataSources: sources,
    intermediateResult: result,
    confidenceScore: confidence,
    timestamp: new Date().toISOString()
  });
  return result;
}

The context-as-goldfish problem is real and underlogged: most tracing setups capture what an agent did but not what it knew when it did it. For short workflows this is acceptable; for long-horizon multi-phase agents it is where the most confusing failures hide. The engineering pattern is to emit a lightweight context manifest at each phase entry — not the full context, but a structured record of which prior phase outputs are still in the active window, which have been summarized or dropped, and which have been offloaded to external memory. This manifest is the thing that makes phase correlation meaningful, because without it you cannot tell whether phase N had access to the result of phase N-3 or whether that result was evicted two phases back.

AIThis connects the IAM logging pattern (block_1), the gate evaluation pattern (block_2), and the tool-call provenance pattern (block_3) into a unified concern: all of them are only fully interpretable if you also know what context the agent was operating with at that moment. A gate failure that looks like a reasoning error might be a context eviction. A credential fetch that looks suspicious might have been the agent re-fetching because its prior result was evicted. The context manifest is the substrate that makes the rest of the provenance record interpretable.

// TypeScript: emit a lightweight context manifest at each phase entry
interface ContextManifest {
  correlationId: string;
  phaseId: string;
  activePhaseResults: string[];  // phase IDs still in active window
  evictedPhaseResults: string[]; // phase IDs dropped since last manifest
  offloadedToMemory: string[];   // phase IDs persisted to external store
  windowTokenEstimate: number;
  timestamp: string;
}

function buildContextManifest(
  correlationId: string,
  phaseId: string,
  previousManifest: ContextManifest | null,
  currentWindowContents: string[],
  memoryStore: Set<string>
): ContextManifest {
  const previousActive = new Set(previousManifest?.activePhaseResults ?? []);
  const currentActive = new Set(currentWindowContents);
  const evicted = [...previousActive].filter(p => !currentActive.has(p) && !memoryStore.has(p));
  const offloaded = [...previousActive].filter(p => !currentActive.has(p) && memoryStore.has(p));

  return {
    correlationId,
    phaseId,
    activePhaseResults: currentWindowContents,
    evictedPhaseResults: evicted,
    offloadedToMemory: offloaded,
    windowTokenEstimate: currentWindowContents.length * 800, // rough estimate
    timestamp: new Date().toISOString()
  };
}