Add Claude observability tracing and diagnostics UI

This commit is contained in:
2026-02-24 12:50:31 -05:00
parent 6863c1da0b
commit 691591d279
22 changed files with 1898 additions and 32 deletions

View File

@@ -16,6 +16,15 @@ CLAUDE_CODE_OAUTH_TOKEN=
ANTHROPIC_API_KEY= ANTHROPIC_API_KEY=
CLAUDE_MODEL= CLAUDE_MODEL=
CLAUDE_CODE_PATH= CLAUDE_CODE_PATH=
# Claude binary observability: off | stdout | file | both
CLAUDE_OBSERVABILITY_MODE=off
# CLAUDE_OBSERVABILITY_VERBOSITY: summary | full
CLAUDE_OBSERVABILITY_VERBOSITY=summary
# Relative to repository workspace root in UI/provider runs.
CLAUDE_OBSERVABILITY_LOG_PATH=.ai_ops/events/claude-trace.ndjson
CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL=false
CLAUDE_OBSERVABILITY_DEBUG=false
CLAUDE_OBSERVABILITY_DEBUG_LOG_PATH=
# Agent management limits # Agent management limits
AGENT_MAX_CONCURRENT=4 AGENT_MAX_CONCURRENT=4

1
.gitignore vendored
View File

@@ -4,3 +4,4 @@ dist
mcp.config.json mcp.config.json
.ai_ops .ai_ops
.agent-context .agent-context
.workspace

View File

@@ -95,6 +95,7 @@ The UI provides:
- graph visualizer with topology/retry rendering, edge trigger labels, node economics (duration/cost/tokens), and critical-path highlighting - graph visualizer with topology/retry rendering, edge trigger labels, node economics (duration/cost/tokens), and critical-path highlighting
- node inspector with attempt metadata and injected `ResolvedExecutionContext` sandbox payload - node inspector with attempt metadata and injected `ResolvedExecutionContext` sandbox payload
- live runtime event feed from `AGENT_RUNTIME_EVENT_LOG_PATH` with severity coloring (including security mirror events) - live runtime event feed from `AGENT_RUNTIME_EVENT_LOG_PATH` with severity coloring (including security mirror events)
- Claude trace feed from `CLAUDE_OBSERVABILITY_LOG_PATH` (query lifecycle, SDK message types/subtypes, and errors)
- run trigger + kill switch backed by `SchemaDrivenExecutionEngine.runSession(...)` - run trigger + kill switch backed by `SchemaDrivenExecutionEngine.runSession(...)`
- run mode selector: `provider` (real Codex/Claude execution) or `mock` (deterministic dry-run executor) - run mode selector: `provider` (real Codex/Claude execution) or `mock` (deterministic dry-run executor)
- provider selector: `codex` or `claude` - provider selector: `codex` or `claude`
@@ -108,6 +109,7 @@ Provider mode notes:
- `provider=codex` uses existing OpenAI/Codex auth settings (`OPENAI_AUTH_MODE`, `CODEX_API_KEY`, `OPENAI_API_KEY`). - `provider=codex` uses existing OpenAI/Codex auth settings (`OPENAI_AUTH_MODE`, `CODEX_API_KEY`, `OPENAI_API_KEY`).
- `provider=claude` uses Claude auth resolution (`CLAUDE_CODE_OAUTH_TOKEN` preferred, otherwise `ANTHROPIC_API_KEY`, or existing Claude Code login state). - `provider=claude` uses Claude auth resolution (`CLAUDE_CODE_OAUTH_TOKEN` preferred, otherwise `ANTHROPIC_API_KEY`, or existing Claude Code login state).
- `CLAUDE_MODEL` should be a Claude model id/alias recognized by Claude Code (for example `claude-sonnet-4-6`); `anthropic/...` prefixes are normalized automatically. - `CLAUDE_MODEL` should be a Claude model id/alias recognized by Claude Code (for example `claude-sonnet-4-6`); `anthropic/...` prefixes are normalized automatically.
- Claude provider runs can emit Claude SDK/CLI internals to stdout and/or NDJSON with `CLAUDE_OBSERVABILITY_*` settings.
## Manifest Semantics ## Manifest Semantics
@@ -202,6 +204,30 @@ Notes:
- `security.tool.invocation_allowed` - `security.tool.invocation_allowed`
- `security.tool.invocation_blocked` - `security.tool.invocation_blocked`
## Claude Observability
- `CLAUDE_OBSERVABILITY_MODE=stdout` prints structured Claude query internals (tool progress, system events, stderr, result lifecycle) to stdout as JSON lines prefixed with `[claude-trace]`.
- `CLAUDE_OBSERVABILITY_MODE=file` appends the same records to `CLAUDE_OBSERVABILITY_LOG_PATH`.
- `CLAUDE_OBSERVABILITY_MODE=both` enables both outputs.
- Output samples high-frequency `tool_progress` events to avoid log flooding while retaining suppression counters.
- `assistant` and `user` message records are retained so turn flow is inspectable end-to-end.
- `CLAUDE_OBSERVABILITY_VERBOSITY=summary` stores compact metadata; `full` stores redacted full SDK message payloads.
- `CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL=true` enables and emits sampled partial assistant stream events from the SDK.
- `CLAUDE_OBSERVABILITY_DEBUG=true` enables Claude SDK debug mode.
- `CLAUDE_OBSERVABILITY_DEBUG_LOG_PATH` writes Claude SDK debug output to a file (also enables debug mode).
- In UI/provider mode, `CLAUDE_OBSERVABILITY_LOG_PATH` resolves relative to the repo workspace root.
- UI API: `GET /api/claude-trace?limit=<n>&sessionId=<id>` reads filtered Claude trace records.
Example:
```bash
CLAUDE_OBSERVABILITY_MODE=both
CLAUDE_OBSERVABILITY_VERBOSITY=summary
CLAUDE_OBSERVABILITY_LOG_PATH=.ai_ops/events/claude-trace.ndjson
CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL=false
CLAUDE_OBSERVABILITY_DEBUG=false
```
### Analytics Quick Start ### Analytics Quick Start
Inspect latest events: Inspect latest events:
@@ -258,6 +284,12 @@ jq -c 'select(.severity=="critical")' .ai_ops/events/runtime-events.ndjson
- `ANTHROPIC_API_KEY` (used when `CLAUDE_CODE_OAUTH_TOKEN` is unset) - `ANTHROPIC_API_KEY` (used when `CLAUDE_CODE_OAUTH_TOKEN` is unset)
- `CLAUDE_MODEL` - `CLAUDE_MODEL`
- `CLAUDE_CODE_PATH` - `CLAUDE_CODE_PATH`
- `CLAUDE_OBSERVABILITY_MODE` (`off`, `stdout`, `file`, or `both`)
- `CLAUDE_OBSERVABILITY_VERBOSITY` (`summary` or `full`)
- `CLAUDE_OBSERVABILITY_LOG_PATH`
- `CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL` (`true` or `false`)
- `CLAUDE_OBSERVABILITY_DEBUG` (`true` or `false`)
- `CLAUDE_OBSERVABILITY_DEBUG_LOG_PATH`
- `MCP_CONFIG_PATH` - `MCP_CONFIG_PATH`
### Agent Manager Limits ### Agent Manager Limits

View File

@@ -190,6 +190,9 @@ function createActorSecurityContext(input: {
type: `security.${event.type}`, type: `security.${event.type}`,
severity: mapSecurityAuditSeverity(event), severity: mapSecurityAuditSeverity(event),
message: toSecurityAuditMessage(event), message: toSecurityAuditMessage(event),
...(event.sessionId ? { sessionId: event.sessionId } : {}),
...(event.nodeId ? { nodeId: event.nodeId } : {}),
...(typeof event.attempt === "number" ? { attempt: event.attempt } : {}),
metadata: toSecurityAuditMetadata(event), metadata: toSecurityAuditMetadata(event),
}); });
}; };

View File

@@ -107,6 +107,8 @@ export type ResolvedExecutionContext = {
export type ActorExecutionInput = { export type ActorExecutionInput = {
sessionId: string; sessionId: string;
attempt: number;
depth: number;
node: PipelineNode; node: PipelineNode;
prompt: string; prompt: string;
context: NodeExecutionContext; context: NodeExecutionContext;
@@ -912,6 +914,8 @@ export class PipelineExecutor {
const result = await this.invokeActorExecutor({ const result = await this.invokeActorExecutor({
sessionId, sessionId,
attempt,
depth: recursiveDepth,
node, node,
prompt, prompt,
context, context,
@@ -1065,6 +1069,8 @@ export class PipelineExecutor {
private async invokeActorExecutor(input: { private async invokeActorExecutor(input: {
sessionId: string; sessionId: string;
attempt: number;
depth: number;
node: PipelineNode; node: PipelineNode;
prompt: string; prompt: string;
context: NodeExecutionContext; context: NodeExecutionContext;
@@ -1077,12 +1083,17 @@ export class PipelineExecutor {
return await input.executor({ return await input.executor({
sessionId: input.sessionId, sessionId: input.sessionId,
attempt: input.attempt,
depth: input.depth,
node: input.node, node: input.node,
prompt: input.prompt, prompt: input.prompt,
context: input.context, context: input.context,
signal: input.signal, signal: input.signal,
executionContext: input.executionContext, executionContext: input.executionContext,
mcp: this.buildActorMcpContext({ mcp: this.buildActorMcpContext({
sessionId: input.sessionId,
nodeId: input.node.id,
attempt: input.attempt,
executionContext: input.executionContext, executionContext: input.executionContext,
prompt: input.prompt, prompt: input.prompt,
}), }),
@@ -1207,6 +1218,9 @@ export class PipelineExecutor {
} }
private buildActorMcpContext(input: { private buildActorMcpContext(input: {
sessionId: string;
nodeId: string;
attempt: number;
executionContext: ResolvedExecutionContext; executionContext: ResolvedExecutionContext;
prompt: string; prompt: string;
}): ActorExecutionMcpContext { }): ActorExecutionMcpContext {
@@ -1261,7 +1275,12 @@ export class PipelineExecutor {
}; };
const createToolPermissionHandler = (): ActorToolPermissionHandler => const createToolPermissionHandler = (): ActorToolPermissionHandler =>
this.createToolPermissionHandler(executionContext.allowedTools); this.createToolPermissionHandler({
allowedTools: executionContext.allowedTools,
sessionId: input.sessionId,
nodeId: input.nodeId,
attempt: input.attempt,
});
return { return {
allowedTools: [...executionContext.allowedTools], allowedTools: [...executionContext.allowedTools],
@@ -1273,10 +1292,20 @@ export class PipelineExecutor {
}; };
} }
private createToolPermissionHandler(allowedTools: readonly string[]): ActorToolPermissionHandler { private createToolPermissionHandler(input: {
const allowset = new Set(allowedTools); allowedTools: readonly string[];
sessionId: string;
nodeId: string;
attempt: number;
}): ActorToolPermissionHandler {
const allowset = new Set(input.allowedTools);
const rulesEngine = this.securityContext?.rulesEngine; const rulesEngine = this.securityContext?.rulesEngine;
const toolPolicy = toAllowedToolPolicy(allowedTools); const toolPolicy = toAllowedToolPolicy(input.allowedTools);
const toolAuditContext = {
sessionId: input.sessionId,
nodeId: input.nodeId,
attempt: input.attempt,
};
return async (toolName, _input, options) => { return async (toolName, _input, options) => {
const toolUseID = options.toolUseID; const toolUseID = options.toolUseID;
@@ -1295,6 +1324,7 @@ export class PipelineExecutor {
rulesEngine?.assertToolInvocationAllowed({ rulesEngine?.assertToolInvocationAllowed({
tool: candidates[0] ?? toolName, tool: candidates[0] ?? toolName,
toolClearance: toolPolicy, toolClearance: toolPolicy,
context: toolAuditContext,
}); });
return { return {
behavior: "deny", behavior: "deny",
@@ -1307,6 +1337,7 @@ export class PipelineExecutor {
rulesEngine?.assertToolInvocationAllowed({ rulesEngine?.assertToolInvocationAllowed({
tool: allowMatch, tool: allowMatch,
toolClearance: toolPolicy, toolClearance: toolPolicy,
context: toolAuditContext,
}); });
return { return {

View File

@@ -76,6 +76,11 @@ type GitExecutionResult = {
stderr: string; stderr: string;
}; };
type GitWorktreeRecord = {
path: string;
branchRef?: string;
};
function toErrorMessage(error: unknown): string { function toErrorMessage(error: unknown): string {
if (error instanceof Error) { if (error instanceof Error) {
return error.message; return error.message;
@@ -198,6 +203,40 @@ function toStringLines(value: string): string[] {
.filter((line) => line.length > 0); .filter((line) => line.length > 0);
} }
function parseGitWorktreeRecords(value: string): GitWorktreeRecord[] {
const lines = value.split("\n");
const records: GitWorktreeRecord[] = [];
let current: GitWorktreeRecord | undefined;
for (const line of lines) {
if (!line.trim()) {
if (current) {
records.push(current);
current = undefined;
}
continue;
}
if (line.startsWith("worktree ")) {
if (current) {
records.push(current);
}
current = {
path: line.slice("worktree ".length).trim(),
};
continue;
}
if (line.startsWith("branch ") && current) {
current.branchRef = line.slice("branch ".length).trim();
}
}
if (current) {
records.push(current);
}
return records;
}
export class FileSystemSessionMetadataStore { export class FileSystemSessionMetadataStore {
private readonly stateRoot: string; private readonly stateRoot: string;
@@ -383,11 +422,44 @@ export class SessionWorktreeManager {
const worktreePath = maybeExisting const worktreePath = maybeExisting
? assertAbsolutePath(maybeExisting, "existingWorktreePath") ? assertAbsolutePath(maybeExisting, "existingWorktreePath")
: this.resolveTaskWorktreePath(input.sessionId, input.taskId); : this.resolveTaskWorktreePath(input.sessionId, input.taskId);
const branchName = this.resolveTaskBranchName(input.sessionId, input.taskId);
const attachedWorktree = await this.findWorktreePathForBranch(baseWorkspacePath, branchName);
if (attachedWorktree && attachedWorktree !== worktreePath) {
throw new Error(
`Task branch "${branchName}" is already attached to worktree "${attachedWorktree}", ` +
`expected "${worktreePath}".`,
);
}
if (!(await pathExists(worktreePath))) {
await runGit(["-C", baseWorkspacePath, "worktree", "prune", "--expire", "now"]);
}
if (!(await pathExists(worktreePath))) { if (!(await pathExists(worktreePath))) {
await mkdir(dirname(worktreePath), { recursive: true }); await mkdir(dirname(worktreePath), { recursive: true });
const branchName = this.resolveTaskBranchName(input.sessionId, input.taskId); const addResult = await runGitWithResult([
await runGit(["-C", baseWorkspacePath, "worktree", "add", "-B", branchName, worktreePath, "HEAD"]); "-C",
baseWorkspacePath,
"worktree",
"add",
"-B",
branchName,
worktreePath,
"HEAD",
]);
if (addResult.exitCode !== 0) {
const attachedAfterFailure = await this.findWorktreePathForBranch(baseWorkspacePath, branchName);
if (attachedAfterFailure === worktreePath && (await pathExists(worktreePath))) {
return {
taskWorktreePath: worktreePath,
};
}
throw new Error(
`git -C ${baseWorkspacePath} worktree add -B ${branchName} ${worktreePath} HEAD failed: ` +
`${toGitFailureMessage(addResult)}`,
);
}
} }
return { return {
@@ -687,4 +759,25 @@ export class SessionWorktreeManager {
const mergeBase = result.stdout.trim(); const mergeBase = result.stdout.trim();
return mergeBase || undefined; return mergeBase || undefined;
} }
private async findWorktreePathForBranch(
repoPath: string,
branchName: string,
): Promise<string | undefined> {
const branchRef = `refs/heads/${branchName}`;
const records = await this.listWorktreeRecords(repoPath);
const matched = records.find((record) => record.branchRef === branchRef);
if (!matched) {
return undefined;
}
return resolve(matched.path);
}
private async listWorktreeRecords(repoPath: string): Promise<GitWorktreeRecord[]> {
const result = await runGitWithResult(["-C", repoPath, "worktree", "list", "--porcelain"]);
if (result.exitCode !== 0) {
return [];
}
return parseGitWorktreeRecords(result.stdout);
}
} }

View File

@@ -16,9 +16,21 @@ export type ProviderRuntimeConfig = {
anthropicApiKey?: string; anthropicApiKey?: string;
claudeModel?: string; claudeModel?: string;
claudeCodePath?: string; claudeCodePath?: string;
claudeObservability: ClaudeObservabilityRuntimeConfig;
}; };
export type OpenAiAuthMode = "auto" | "chatgpt" | "api_key"; export type OpenAiAuthMode = "auto" | "chatgpt" | "api_key";
export type ClaudeObservabilityMode = "off" | "stdout" | "file" | "both";
export type ClaudeObservabilityVerbosity = "summary" | "full";
export type ClaudeObservabilityRuntimeConfig = {
mode: ClaudeObservabilityMode;
verbosity: ClaudeObservabilityVerbosity;
logPath: string;
includePartialMessages: boolean;
debug: boolean;
debugLogPath?: string;
};
export type McpRuntimeConfig = { export type McpRuntimeConfig = {
configPath: string; configPath: string;
@@ -115,6 +127,15 @@ const DEFAULT_RUNTIME_EVENTS: RuntimeEventRuntimeConfig = {
discordAlwaysNotifyTypes: ["session.started", "session.completed", "session.failed"], discordAlwaysNotifyTypes: ["session.started", "session.completed", "session.failed"],
}; };
const DEFAULT_CLAUDE_OBSERVABILITY: ClaudeObservabilityRuntimeConfig = {
mode: "off",
verbosity: "summary",
logPath: ".ai_ops/events/claude-trace.ndjson",
includePartialMessages: false,
debug: false,
debugLogPath: undefined,
};
function readOptionalString( function readOptionalString(
env: NodeJS.ProcessEnv, env: NodeJS.ProcessEnv,
key: string, key: string,
@@ -274,6 +295,26 @@ function parseOpenAiAuthMode(raw: string): OpenAiAuthMode {
); );
} }
function parseClaudeObservabilityMode(raw: string): ClaudeObservabilityMode {
if (raw === "off" || raw === "stdout" || raw === "file" || raw === "both") {
return raw;
}
throw new Error(
'Environment variable CLAUDE_OBSERVABILITY_MODE must be one of: "off", "stdout", "file", "both".',
);
}
function parseClaudeObservabilityVerbosity(raw: string): ClaudeObservabilityVerbosity {
if (raw === "summary" || raw === "full") {
return raw;
}
throw new Error(
'Environment variable CLAUDE_OBSERVABILITY_VERBOSITY must be one of: "summary", "full".',
);
}
function deepFreeze<T>(value: T): Readonly<T> { function deepFreeze<T>(value: T): Readonly<T> {
if (value === null || typeof value !== "object") { if (value === null || typeof value !== "object") {
return value; return value;
@@ -360,6 +401,38 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Readonly<AppCo
anthropicApiKey, anthropicApiKey,
claudeModel: normalizeClaudeModel(readOptionalString(env, "CLAUDE_MODEL")), claudeModel: normalizeClaudeModel(readOptionalString(env, "CLAUDE_MODEL")),
claudeCodePath: readOptionalString(env, "CLAUDE_CODE_PATH"), claudeCodePath: readOptionalString(env, "CLAUDE_CODE_PATH"),
claudeObservability: {
mode: parseClaudeObservabilityMode(
readStringWithFallback(
env,
"CLAUDE_OBSERVABILITY_MODE",
DEFAULT_CLAUDE_OBSERVABILITY.mode,
),
),
verbosity: parseClaudeObservabilityVerbosity(
readStringWithFallback(
env,
"CLAUDE_OBSERVABILITY_VERBOSITY",
DEFAULT_CLAUDE_OBSERVABILITY.verbosity,
),
),
logPath: readStringWithFallback(
env,
"CLAUDE_OBSERVABILITY_LOG_PATH",
DEFAULT_CLAUDE_OBSERVABILITY.logPath,
),
includePartialMessages: readBooleanWithFallback(
env,
"CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL",
DEFAULT_CLAUDE_OBSERVABILITY.includePartialMessages,
),
debug: readBooleanWithFallback(
env,
"CLAUDE_OBSERVABILITY_DEBUG",
DEFAULT_CLAUDE_OBSERVABILITY.debug,
),
debugLogPath: readOptionalString(env, "CLAUDE_OBSERVABILITY_DEBUG_LOG_PATH"),
},
}, },
mcp: { mcp: {
configPath: readStringWithFallback(env, "MCP_CONFIG_PATH", "./mcp.config.json"), configPath: readStringWithFallback(env, "MCP_CONFIG_PATH", "./mcp.config.json"),

View File

@@ -13,41 +13,43 @@ import {
} from "./schemas.js"; } from "./schemas.js";
export type SecurityAuditEvent = export type SecurityAuditEvent =
| { | ({
type: "shell.command_profiled"; type: "shell.command_profiled";
timestamp: string;
command: string; command: string;
cwd: string; cwd: string;
parsed: ParsedShellScript; parsed: ParsedShellScript;
} } & SecurityAuditContext)
| { | ({
type: "shell.command_allowed"; type: "shell.command_allowed";
timestamp: string;
command: string; command: string;
cwd: string; cwd: string;
commandCount: number; commandCount: number;
} } & SecurityAuditContext)
| { | ({
type: "shell.command_blocked"; type: "shell.command_blocked";
timestamp: string;
command: string; command: string;
cwd: string; cwd: string;
reason: string; reason: string;
code: string; code: string;
details?: Record<string, unknown>; details?: Record<string, unknown>;
} } & SecurityAuditContext)
| { | ({
type: "tool.invocation_allowed"; type: "tool.invocation_allowed";
timestamp: string;
tool: string; tool: string;
} } & SecurityAuditContext)
| { | ({
type: "tool.invocation_blocked"; type: "tool.invocation_blocked";
timestamp: string;
tool: string; tool: string;
reason: string; reason: string;
code: string; code: string;
}; } & SecurityAuditContext);
export type SecurityAuditContext = {
timestamp: string;
sessionId?: string;
nodeId?: string;
attempt?: number;
};
export type SecurityAuditSink = (event: SecurityAuditEvent) => void; export type SecurityAuditSink = (event: SecurityAuditEvent) => void;
@@ -102,6 +104,28 @@ function toNow(): string {
return new Date().toISOString(); return new Date().toISOString();
} }
function toAuditContext(input?: {
sessionId?: string;
nodeId?: string;
attempt?: number;
}): SecurityAuditContext {
const output: SecurityAuditContext = {
timestamp: toNow(),
};
if (input?.sessionId) {
output.sessionId = input.sessionId;
}
if (input?.nodeId) {
output.nodeId = input.nodeId;
}
if (typeof input?.attempt === "number" && Number.isInteger(input.attempt) && input.attempt >= 1) {
output.attempt = input.attempt;
}
return output;
}
export class SecurityRulesEngine { export class SecurityRulesEngine {
private readonly policy: ShellValidationPolicy; private readonly policy: ShellValidationPolicy;
private readonly allowedBinaries: Set<string>; private readonly allowedBinaries: Set<string>;
@@ -136,6 +160,11 @@ export class SecurityRulesEngine {
command: string; command: string;
cwd: string; cwd: string;
toolClearance?: ToolClearancePolicy; toolClearance?: ToolClearancePolicy;
context?: {
sessionId?: string;
nodeId?: string;
attempt?: number;
};
}): Promise<ValidatedShellCommand> { }): Promise<ValidatedShellCommand> {
const resolvedCwd = resolve(input.cwd); const resolvedCwd = resolve(input.cwd);
@@ -147,22 +176,22 @@ export class SecurityRulesEngine {
: undefined; : undefined;
this.emit({ this.emit({
...toAuditContext(input.context),
type: "shell.command_profiled", type: "shell.command_profiled",
timestamp: toNow(),
command: input.command, command: input.command,
cwd: resolvedCwd, cwd: resolvedCwd,
parsed, parsed,
}); });
for (const command of parsed.commands) { for (const command of parsed.commands) {
this.assertBinaryAllowed(command, toolClearance); this.assertBinaryAllowed(command, toolClearance, input.context);
this.assertAssignmentsAllowed(command); this.assertAssignmentsAllowed(command);
this.assertArgumentPaths(command, resolvedCwd); this.assertArgumentPaths(command, resolvedCwd);
} }
this.emit({ this.emit({
...toAuditContext(input.context),
type: "shell.command_allowed", type: "shell.command_allowed",
timestamp: toNow(),
command: input.command, command: input.command,
cwd: resolvedCwd, cwd: resolvedCwd,
commandCount: parsed.commandCount, commandCount: parsed.commandCount,
@@ -175,8 +204,8 @@ export class SecurityRulesEngine {
} catch (error) { } catch (error) {
if (error instanceof SecurityViolationError) { if (error instanceof SecurityViolationError) {
this.emit({ this.emit({
...toAuditContext(input.context),
type: "shell.command_blocked", type: "shell.command_blocked",
timestamp: toNow(),
command: input.command, command: input.command,
cwd: resolvedCwd, cwd: resolvedCwd,
reason: error.message, reason: error.message,
@@ -196,13 +225,18 @@ export class SecurityRulesEngine {
assertToolInvocationAllowed(input: { assertToolInvocationAllowed(input: {
tool: string; tool: string;
toolClearance: ToolClearancePolicy; toolClearance: ToolClearancePolicy;
context?: {
sessionId?: string;
nodeId?: string;
attempt?: number;
};
}): void { }): void {
const policy = parseToolClearancePolicy(input.toolClearance); const policy = parseToolClearancePolicy(input.toolClearance);
if (policy.banlist.includes(input.tool)) { if (policy.banlist.includes(input.tool)) {
this.emit({ this.emit({
...toAuditContext(input.context),
type: "tool.invocation_blocked", type: "tool.invocation_blocked",
timestamp: toNow(),
tool: input.tool, tool: input.tool,
reason: `Tool "${input.tool}" is explicitly banned by policy.`, reason: `Tool "${input.tool}" is explicitly banned by policy.`,
code: "TOOL_BANNED", code: "TOOL_BANNED",
@@ -220,8 +254,8 @@ export class SecurityRulesEngine {
if (policy.allowlist.length > 0 && !policy.allowlist.includes(input.tool)) { if (policy.allowlist.length > 0 && !policy.allowlist.includes(input.tool)) {
this.emit({ this.emit({
...toAuditContext(input.context),
type: "tool.invocation_blocked", type: "tool.invocation_blocked",
timestamp: toNow(),
tool: input.tool, tool: input.tool,
reason: `Tool "${input.tool}" is not present in allowlist.`, reason: `Tool "${input.tool}" is not present in allowlist.`,
code: "TOOL_NOT_ALLOWED", code: "TOOL_NOT_ALLOWED",
@@ -238,8 +272,8 @@ export class SecurityRulesEngine {
} }
this.emit({ this.emit({
...toAuditContext(input.context),
type: "tool.invocation_allowed", type: "tool.invocation_allowed",
timestamp: toNow(),
tool: input.tool, tool: input.tool,
}); });
} }
@@ -290,6 +324,11 @@ export class SecurityRulesEngine {
private assertBinaryAllowed( private assertBinaryAllowed(
command: ParsedShellCommand, command: ParsedShellCommand,
toolClearance?: ToolClearancePolicy, toolClearance?: ToolClearancePolicy,
context?: {
sessionId?: string;
nodeId?: string;
attempt?: number;
},
): void { ): void {
const binaryToken = normalizeToken(command.binary); const binaryToken = normalizeToken(command.binary);
const binaryName = basename(binaryToken); const binaryName = basename(binaryToken);
@@ -313,6 +352,7 @@ export class SecurityRulesEngine {
this.assertToolInvocationAllowed({ this.assertToolInvocationAllowed({
tool: binaryName, tool: binaryName,
toolClearance, toolClearance,
context,
}); });
} }

View File

@@ -0,0 +1,821 @@
import { randomUUID } from "node:crypto";
import { appendFile, mkdir } from "node:fs/promises";
import { dirname, resolve } from "node:path";
import type { Options, SDKMessage } from "@anthropic-ai/claude-agent-sdk";
import type {
ClaudeObservabilityMode,
ClaudeObservabilityRuntimeConfig,
ClaudeObservabilityVerbosity,
} from "../config.js";
import type { JsonObject, JsonValue } from "../agents/types.js";
const MAX_STRING_LENGTH = 320;
const MAX_ARRAY_ITEMS = 20;
const MAX_OBJECT_KEYS = 60;
const MAX_DEPTH = 6;
const NON_SECRET_TOKEN_KEYS = new Set([
"input_tokens",
"output_tokens",
"total_tokens",
"cache_creation_input_tokens",
"cache_read_input_tokens",
"ephemeral_1h_input_tokens",
"ephemeral_5m_input_tokens",
"token_input",
"token_output",
"token_total",
"tokencount",
"token_count",
"tool_use_id",
"parent_tool_use_id",
"task_id",
"session_id",
]);
type ClaudeTraceContext = {
sessionId: string;
nodeId: string;
attempt: number;
depth: number;
};
type ClaudeTraceRecord = {
id: string;
timestamp: string;
source: "claude_sdk";
stage:
| "query.started"
| "query.message"
| "query.stderr"
| "query.completed"
| "query.error";
message: string;
sessionId: string;
nodeId: string;
attempt: number;
depth: number;
sdkSessionId?: string;
sdkMessageType?: string;
sdkMessageSubtype?: string;
data?: JsonObject;
};
function truncate(value: string, maxLength = MAX_STRING_LENGTH): string {
if (value.length <= maxLength) {
return value;
}
return `${value.slice(0, maxLength)}...`;
}
function isSensitiveKey(key: string): boolean {
const normalized = key.trim().toLowerCase();
if (!normalized) {
return false;
}
if (NON_SECRET_TOKEN_KEYS.has(normalized)) {
return false;
}
if (/(api[_-]?key|secret|password|authorization|cookie)/i.test(key)) {
return true;
}
if (/(auth[_-]?token|access[_-]?token|refresh[_-]?token|id[_-]?token|oauth)/i.test(key)) {
return true;
}
return normalized === "token";
}
function toJsonPrimitive(value: unknown): JsonValue {
if (value === null) {
return null;
}
if (typeof value === "string") {
return truncate(value);
}
if (typeof value === "number") {
return Number.isFinite(value) ? value : String(value);
}
if (typeof value === "boolean") {
return value;
}
if (typeof value === "bigint") {
return String(value);
}
if (typeof value === "undefined") {
return null;
}
return truncate(String(value));
}
function sanitizeJsonValue(value: unknown, depth = 0): JsonValue {
if (depth >= MAX_DEPTH) {
return "[depth_limit]";
}
if (
value === null ||
typeof value === "string" ||
typeof value === "number" ||
typeof value === "boolean" ||
typeof value === "bigint" ||
typeof value === "undefined"
) {
return toJsonPrimitive(value);
}
if (Array.isArray(value)) {
const output = value.slice(0, MAX_ARRAY_ITEMS).map((entry) => sanitizeJsonValue(entry, depth + 1));
if (value.length > MAX_ARRAY_ITEMS) {
output.push(`[+${String(value.length - MAX_ARRAY_ITEMS)} more]`);
}
return output;
}
if (typeof value === "object") {
const output: JsonObject = {};
const entries = Object.entries(value as Record<string, unknown>);
const limited = entries.slice(0, MAX_OBJECT_KEYS);
for (const [key, entryValue] of limited) {
if (isSensitiveKey(key)) {
output[key] = "[redacted]";
continue;
}
output[key] = sanitizeJsonValue(entryValue, depth + 1);
}
if (entries.length > MAX_OBJECT_KEYS) {
output.__truncated_keys = entries.length - MAX_OBJECT_KEYS;
}
return output;
}
return truncate(String(value));
}
function readString(value: unknown): string | undefined {
if (typeof value !== "string") {
return undefined;
}
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : undefined;
}
function readNumber(value: unknown): number | undefined {
return typeof value === "number" && Number.isFinite(value) ? value : undefined;
}
function readBoolean(value: unknown): boolean | undefined {
return typeof value === "boolean" ? value : undefined;
}
function toMessageRecord(message: SDKMessage): Record<string, unknown> {
return message as unknown as Record<string, unknown>;
}
function toMessageSubtype(message: SDKMessage): string | undefined {
return readString(toMessageRecord(message).subtype);
}
function toMessageSessionId(message: SDKMessage): string | undefined {
return readString(toMessageRecord(message).session_id);
}
function toTaskNotificationSummary(message: SDKMessage): {
summary: string;
data?: JsonObject;
} {
const raw = toMessageRecord(message);
const status = readString(raw.status) ?? "unknown";
const data: JsonObject = {
status,
};
const taskId = readString(raw.task_id);
if (taskId) {
data.taskId = taskId;
}
const summaryText = readString(raw.summary);
if (summaryText) {
data.summary = truncate(summaryText);
}
const outputFile = readString(raw.output_file);
if (outputFile) {
data.outputFile = outputFile;
}
if (raw.usage !== undefined) {
data.usage = sanitizeJsonValue(raw.usage);
}
return {
summary: `Task notification: ${status}.`,
data,
};
}
function toTaskStartedSummary(message: SDKMessage): {
summary: string;
data?: JsonObject;
} {
const raw = toMessageRecord(message);
const data: JsonObject = {};
const taskId = readString(raw.task_id);
if (taskId) {
data.taskId = taskId;
}
const description = readString(raw.description);
if (description) {
data.description = truncate(description);
}
const taskType = readString(raw.task_type);
if (taskType) {
data.taskType = taskType;
}
const toolUseId = readString(raw.tool_use_id);
if (toolUseId) {
data.toolUseId = toolUseId;
}
return {
summary: "Task started.",
...(Object.keys(data).length > 0 ? { data } : {}),
};
}
function toMessageSummary(message: SDKMessage): {
summary: string;
data?: JsonObject;
} {
const subtype = toMessageSubtype(message);
const raw = toMessageRecord(message);
if (message.type === "result") {
if (message.subtype === "success") {
return {
summary: "Claude query result success.",
data: {
stopReason: message.stop_reason ?? null,
numTurns: message.num_turns,
usage: sanitizeJsonValue(message.usage) as JsonObject,
totalCostUsd: message.total_cost_usd,
},
};
}
return {
summary: `Claude query result ${message.subtype}.`,
data: {
stopReason: message.stop_reason ?? null,
numTurns: message.num_turns,
usage: sanitizeJsonValue(message.usage) as JsonObject,
totalCostUsd: message.total_cost_usd,
errors: sanitizeJsonValue(message.errors),
},
};
}
if (message.type === "tool_progress") {
return {
summary: `Tool progress: ${message.tool_name}.`,
data: {
toolName: message.tool_name,
toolUseId: message.tool_use_id,
elapsedTimeSeconds: message.elapsed_time_seconds,
parentToolUseId: message.parent_tool_use_id ?? null,
...(message.task_id ? { taskId: message.task_id } : {}),
},
};
}
if (message.type === "tool_use_summary") {
return {
summary: "Tool use summary emitted.",
data: {
summary: truncate(message.summary),
precedingToolUseIds: sanitizeJsonValue(message.preceding_tool_use_ids),
},
};
}
if (message.type === "stream_event") {
const data: JsonObject = {};
const eventType = readString((raw.event as Record<string, unknown> | undefined)?.type);
if (eventType) {
data.eventType = eventType;
}
const parentToolUseId = readString(raw.parent_tool_use_id);
if (parentToolUseId) {
data.parentToolUseId = parentToolUseId;
}
return {
summary: "Partial assistant stream event emitted.",
...(Object.keys(data).length > 0 ? { data } : {}),
};
}
if (message.type === "auth_status") {
return {
summary: message.isAuthenticating ? "Authentication in progress." : "Authentication status update.",
data: {
isAuthenticating: message.isAuthenticating,
output: sanitizeJsonValue(message.output),
...(message.error ? { error: truncate(message.error) } : {}),
},
};
}
if (message.type === "assistant") {
return {
summary: "Assistant message emitted.",
data: {
parentToolUseId: message.parent_tool_use_id ?? null,
...(message.error ? { error: message.error } : {}),
},
};
}
if (message.type === "user") {
const data: JsonObject = {
parentToolUseId: (message as { parent_tool_use_id?: string | null }).parent_tool_use_id ?? null,
};
const isSynthetic = readBoolean(raw.isSynthetic);
if (isSynthetic !== undefined) {
data.isSynthetic = isSynthetic;
}
const isReplay = readBoolean(raw.isReplay);
if (isReplay !== undefined) {
data.isReplay = isReplay;
}
return {
summary: "User message emitted.",
data,
};
}
if (subtype === "task_notification") {
return toTaskNotificationSummary(message);
}
if (subtype === "task_started") {
return toTaskStartedSummary(message);
}
if (message.type === "system" && subtype === "files_persisted") {
const files = Array.isArray(raw.files) ? raw.files : [];
const failed = Array.isArray(raw.failed) ? raw.failed : [];
return {
summary: "System event: files_persisted.",
data: {
persistedFileCount: files.length,
failedFileCount: failed.length,
},
};
}
if (message.type === "system" && subtype === "compact_boundary") {
return {
summary: "System event: compact_boundary.",
data: {
compactMetadata: sanitizeJsonValue(raw.compact_metadata),
},
};
}
if (message.type === "system" && subtype === "status") {
const data: JsonObject = {
status: readString(raw.status) ?? "none",
};
const permissionMode = readString(raw.permissionMode);
if (permissionMode) {
data.permissionMode = permissionMode;
}
return {
summary: "System event: status.",
data,
};
}
if (message.type === "system" && (subtype === "hook_started" || subtype === "hook_progress" || subtype === "hook_response")) {
const data: JsonObject = {
...(subtype ? { subtype } : {}),
...(readString(raw.hook_id) ? { hookId: readString(raw.hook_id) } : {}),
...(readString(raw.hook_name) ? { hookName: readString(raw.hook_name) } : {}),
...(readString(raw.hook_event) ? { hookEvent: readString(raw.hook_event) } : {}),
...(readString(raw.outcome) ? { outcome: readString(raw.outcome) } : {}),
};
if (raw.exit_code !== undefined) {
data.exitCode = sanitizeJsonValue(raw.exit_code);
}
return {
summary: `System event: ${subtype}.`,
data,
};
}
if (message.type === "system") {
return {
summary: subtype ? `System event: ${subtype}.` : "System event emitted.",
data: subtype ? { subtype } : undefined,
};
}
if (message.type === "rate_limit") {
return {
summary: "Rate limit event emitted.",
data: sanitizeJsonValue(raw) as JsonObject,
};
}
if (message.type === "prompt_suggestion") {
const data: JsonObject = {
...(readString(raw.prompt) ? { prompt: truncate(readString(raw.prompt) as string) } : {}),
...(readString(raw.suggestion) ? { suggestion: truncate(readString(raw.suggestion) as string) } : {}),
};
return {
summary: "Prompt suggestion emitted.",
...(Object.keys(data).length > 0 ? { data } : {}),
};
}
return {
summary: `Claude SDK message received (${message.type}).`,
};
}
function toRecord(input: {
stage: ClaudeTraceRecord["stage"];
message: string;
context: ClaudeTraceContext;
sdkMessageType?: string;
sdkMessageSubtype?: string;
sdkSessionId?: string;
data?: JsonObject;
}): ClaudeTraceRecord {
return {
id: randomUUID(),
timestamp: new Date().toISOString(),
source: "claude_sdk",
stage: input.stage,
message: input.message,
sessionId: input.context.sessionId,
nodeId: input.context.nodeId,
attempt: input.context.attempt,
depth: input.context.depth,
...(input.sdkMessageType ? { sdkMessageType: input.sdkMessageType } : {}),
...(input.sdkMessageSubtype ? { sdkMessageSubtype: input.sdkMessageSubtype } : {}),
...(input.sdkSessionId ? { sdkSessionId: input.sdkSessionId } : {}),
...(input.data ? { data: input.data } : {}),
};
}
export function summarizeClaudeMessage(
message: SDKMessage,
verbosity: ClaudeObservabilityVerbosity,
): {
messageType: string;
messageSubtype?: string;
sdkSessionId?: string;
summary: string;
data?: JsonObject;
} {
const messageSubtype = toMessageSubtype(message);
const sdkSessionId = toMessageSessionId(message);
const summary = toMessageSummary(message);
if (verbosity === "full") {
return {
messageType: message.type,
...(messageSubtype ? { messageSubtype } : {}),
...(sdkSessionId ? { sdkSessionId } : {}),
summary: summary.summary,
data: {
message: sanitizeJsonValue(message) as JsonObject,
},
};
}
return {
messageType: message.type,
...(messageSubtype ? { messageSubtype } : {}),
...(sdkSessionId ? { sdkSessionId } : {}),
summary: summary.summary,
...(summary.data ? { data: summary.data } : {}),
};
}
export class ClaudeObservabilityLogger {
private readonly mode: ClaudeObservabilityMode;
private readonly verbosity: ClaudeObservabilityVerbosity;
private readonly logPath: string;
private readonly includePartialMessages: boolean;
private readonly debug: boolean;
private readonly debugLogPath?: string;
private readonly pendingWrites = new Set<Promise<void>>();
private readonly stdoutProgressByKey = new Map<string, {
lastEmittedAt: number;
suppressed: number;
}>();
private readonly fileProgressByKey = new Map<string, {
lastEmittedAt: number;
suppressed: number;
}>();
private readonly stdoutStreamByKey = new Map<string, {
lastEmittedAt: number;
suppressed: number;
}>();
private readonly fileStreamByKey = new Map<string, {
lastEmittedAt: number;
suppressed: number;
}>();
private fileWriteFailureCount = 0;
constructor(input: {
workspaceRoot: string;
config: ClaudeObservabilityRuntimeConfig;
}) {
this.mode = input.config.mode;
this.verbosity = input.config.verbosity;
this.logPath = resolve(input.workspaceRoot, input.config.logPath);
this.includePartialMessages = input.config.includePartialMessages;
this.debug = input.config.debug;
this.debugLogPath = input.config.debugLogPath
? resolve(input.workspaceRoot, input.config.debugLogPath)
: undefined;
}
isEnabled(): boolean {
return this.mode !== "off";
}
toOptionOverrides(input: {
context: ClaudeTraceContext;
}): Pick<Options, "includePartialMessages" | "debug" | "debugFile" | "stderr"> {
return {
includePartialMessages: this.includePartialMessages,
debug: this.debug || this.debugLogPath !== undefined,
...(this.debugLogPath ? { debugFile: this.debugLogPath } : {}),
stderr: (data: string): void => {
this.record({
stage: "query.stderr",
message: "Claude SDK stderr output.",
context: input.context,
data: {
stderr: sanitizeJsonValue(data),
},
});
},
};
}
recordQueryStarted(input: {
context: ClaudeTraceContext;
data?: JsonObject;
}): void {
this.record({
stage: "query.started",
message: "Claude query started.",
context: input.context,
...(input.data ? { data: input.data } : {}),
});
}
recordMessage(input: {
context: ClaudeTraceContext;
message: SDKMessage;
}): void {
const summarized = summarizeClaudeMessage(input.message, this.verbosity);
this.record({
stage: "query.message",
message: summarized.summary,
context: input.context,
sdkMessageType: summarized.messageType,
sdkMessageSubtype: summarized.messageSubtype,
sdkSessionId: summarized.sdkSessionId,
...(summarized.data ? { data: summarized.data } : {}),
});
}
recordQueryCompleted(input: {
context: ClaudeTraceContext;
data?: JsonObject;
}): void {
this.record({
stage: "query.completed",
message: "Claude query completed.",
context: input.context,
...(input.data ? { data: input.data } : {}),
});
}
recordQueryError(input: {
context: ClaudeTraceContext;
error: unknown;
}): void {
const errorMessage = input.error instanceof Error ? input.error.message : String(input.error);
this.record({
stage: "query.error",
message: "Claude query failed.",
context: input.context,
data: {
error: truncate(errorMessage),
},
});
}
async close(): Promise<void> {
await Promise.all([...this.pendingWrites]);
}
private record(input: {
stage: ClaudeTraceRecord["stage"];
message: string;
context: ClaudeTraceContext;
sdkMessageType?: string;
sdkMessageSubtype?: string;
sdkSessionId?: string;
data?: JsonObject;
}): void {
if (!this.isEnabled()) {
return;
}
const record = toRecord(input);
if (this.mode === "stdout" || this.mode === "both") {
const stdoutRecord = this.toStdoutRecord(record);
if (stdoutRecord) {
console.log(`[claude-trace] ${JSON.stringify(stdoutRecord)}`);
}
}
if (this.mode === "file" || this.mode === "both") {
const fileRecord = this.toFileRecord(record);
if (!fileRecord) {
return;
}
const line = JSON.stringify(fileRecord);
const write = mkdir(dirname(this.logPath), { recursive: true })
.then(() => appendFile(this.logPath, `${line}\n`, "utf8"))
.catch((error: unknown) => {
this.reportFileWriteFailure(error);
})
.finally(() => {
this.pendingWrites.delete(write);
});
this.pendingWrites.add(write);
}
}
private toStdoutRecord(record: ClaudeTraceRecord): ClaudeTraceRecord | undefined {
return this.toFilteredMessageRecord(record, "stdout");
}
private toFileRecord(record: ClaudeTraceRecord): ClaudeTraceRecord | undefined {
return this.toFilteredMessageRecord(record, "file");
}
private toFilteredMessageRecord(
record: ClaudeTraceRecord,
destination: "stdout" | "file",
): ClaudeTraceRecord | undefined {
if (record.stage !== "query.message") {
return record;
}
if (!record.sdkMessageType) {
return record;
}
if (record.sdkMessageType === "tool_progress") {
return this.toSampledToolProgressRecord(record, destination);
}
if (record.sdkMessageType === "stream_event") {
if (!this.includePartialMessages) {
return undefined;
}
return this.toSampledStreamEventRecord(record, destination);
}
if (record.sdkMessageType === "auth_status") {
const data = record.data;
const isAuthenticating = data?.isAuthenticating === true;
const hasError = typeof data?.error === "string" && data.error.trim().length > 0;
if (hasError || !isAuthenticating) {
return record;
}
return undefined;
}
return record;
}
private toSampledToolProgressRecord(
record: ClaudeTraceRecord,
destination: "stdout" | "file",
): ClaudeTraceRecord | undefined {
const now = Date.now();
const minIntervalMs = destination === "stdout" ? 1000 : 2000;
const rawToolUseId = record.data?.toolUseId;
const toolUseId = typeof rawToolUseId === "string" ? rawToolUseId : "unknown";
const key = `${destination}:${record.sessionId}:${record.nodeId}:${toolUseId}`;
const progressByKey = destination === "stdout" ? this.stdoutProgressByKey : this.fileProgressByKey;
const state = progressByKey.get(key);
if (!state) {
progressByKey.set(key, {
lastEmittedAt: now,
suppressed: 0,
});
return record;
}
if (now - state.lastEmittedAt < minIntervalMs) {
state.suppressed += 1;
return undefined;
}
state.lastEmittedAt = now;
const suppressed = state.suppressed;
state.suppressed = 0;
if (suppressed < 1) {
return record;
}
const nextData: JsonObject = {
...(record.data ?? {}),
suppressedSinceLastEmit: suppressed,
};
return {
...record,
data: nextData,
};
}
private toSampledStreamEventRecord(
record: ClaudeTraceRecord,
destination: "stdout" | "file",
): ClaudeTraceRecord | undefined {
const now = Date.now();
const minIntervalMs = destination === "stdout" ? 700 : 1200;
const key = `${destination}:${record.sessionId}:${record.nodeId}:stream`;
const streamByKey = destination === "stdout" ? this.stdoutStreamByKey : this.fileStreamByKey;
const state = streamByKey.get(key);
if (!state) {
streamByKey.set(key, {
lastEmittedAt: now,
suppressed: 0,
});
return record;
}
if (now - state.lastEmittedAt < minIntervalMs) {
state.suppressed += 1;
return undefined;
}
state.lastEmittedAt = now;
const suppressed = state.suppressed;
state.suppressed = 0;
if (suppressed < 1) {
return record;
}
const nextData: JsonObject = {
...(record.data ?? {}),
suppressedStreamEventsSinceLastEmit: suppressed,
};
return {
...record,
data: nextData,
};
}
private reportFileWriteFailure(error: unknown): void {
this.fileWriteFailureCount += 1;
if (this.fileWriteFailureCount <= 5) {
const message = error instanceof Error ? error.message : String(error);
console.warn(
`[claude-trace] failed to append trace log to ${this.logPath}: ${truncate(message, 180)}`,
);
return;
}
if (this.fileWriteFailureCount === 6) {
console.warn("[claude-trace] additional trace-log write failures suppressed.");
}
}
}

View File

@@ -0,0 +1,85 @@
import { readFile } from "node:fs/promises";
import { resolve } from "node:path";
export type ClaudeTraceEvent = {
timestamp: string;
message: string;
stage?: string;
sessionId?: string;
sdkMessageType?: string;
sdkMessageSubtype?: string;
data?: unknown;
} & Record<string, unknown>;
type ClaudeTraceFilter = {
sessionId?: string;
limit?: number;
};
function safeParseLine(line: string): ClaudeTraceEvent | undefined {
const trimmed = line.trim();
if (!trimmed) {
return undefined;
}
try {
const parsed = JSON.parse(trimmed) as unknown;
if (!parsed || typeof parsed !== "object") {
return undefined;
}
const record = parsed as Record<string, unknown>;
if (typeof record.timestamp !== "string" || typeof record.message !== "string") {
return undefined;
}
return record as ClaudeTraceEvent;
} catch {
return undefined;
}
}
export async function readClaudeTraceEvents(logPath: string): Promise<ClaudeTraceEvent[]> {
const absolutePath = resolve(logPath);
let content = "";
try {
content = await readFile(absolutePath, "utf8");
} catch (error) {
if ((error as NodeJS.ErrnoException).code === "ENOENT") {
return [];
}
throw error;
}
const parsed: ClaudeTraceEvent[] = [];
for (const line of content.split(/\r?\n/)) {
const event = safeParseLine(line);
if (event) {
parsed.push(event);
}
}
parsed.sort((left, right) => left.timestamp.localeCompare(right.timestamp));
return parsed;
}
export function filterClaudeTraceEvents(
events: readonly ClaudeTraceEvent[],
filter: ClaudeTraceFilter,
): ClaudeTraceEvent[] {
const filtered: ClaudeTraceEvent[] = [];
for (const event of events) {
if (filter.sessionId && event.sessionId !== filter.sessionId) {
continue;
}
filtered.push(event);
}
if (!filter.limit || filter.limit < 1 || filtered.length <= filter.limit) {
return filtered;
}
return filtered.slice(-filter.limit);
}

View File

@@ -39,6 +39,7 @@ export type UiConfigSnapshot = {
stateRoot: string; stateRoot: string;
projectContextPath: string; projectContextPath: string;
runtimeEventLogPath: string; runtimeEventLogPath: string;
claudeTraceLogPath: string;
securityAuditLogPath: string; securityAuditLogPath: string;
}; };
}; };
@@ -107,6 +108,7 @@ function toSnapshot(config: Readonly<AppConfig>, envFilePath: string): UiConfigS
stateRoot: config.orchestration.stateRoot, stateRoot: config.orchestration.stateRoot,
projectContextPath: config.orchestration.projectContextPath, projectContextPath: config.orchestration.projectContextPath,
runtimeEventLogPath: config.runtimeEvents.logPath, runtimeEventLogPath: config.runtimeEvents.logPath,
claudeTraceLogPath: config.provider.claudeObservability.logPath,
securityAuditLogPath: config.security.auditLogPath, securityAuditLogPath: config.security.auditLogPath,
}, },
}; };

View File

@@ -10,6 +10,7 @@ import { isDomainEventType, type DomainEventEmission } from "../agents/domain-ev
import type { ActorExecutionInput, ActorExecutionResult, ActorExecutor } from "../agents/pipeline.js"; import type { ActorExecutionInput, ActorExecutionResult, ActorExecutor } from "../agents/pipeline.js";
import { isRecord, type JsonObject, type JsonValue } from "../agents/types.js"; import { isRecord, type JsonObject, type JsonValue } from "../agents/types.js";
import { createSessionContext, type SessionContext } from "../examples/session-context.js"; import { createSessionContext, type SessionContext } from "../examples/session-context.js";
import { ClaudeObservabilityLogger } from "./claude-observability.js";
export type RunProvider = "codex" | "claude"; export type RunProvider = "codex" | "claude";
@@ -17,6 +18,7 @@ export type ProviderRunRuntime = {
provider: RunProvider; provider: RunProvider;
config: Readonly<AppConfig>; config: Readonly<AppConfig>;
sessionContext: SessionContext; sessionContext: SessionContext;
claudeObservability: ClaudeObservabilityLogger;
close: () => Promise<void>; close: () => Promise<void>;
}; };
@@ -416,6 +418,40 @@ type ClaudeTurnResult = {
usage: ProviderUsage; usage: ProviderUsage;
}; };
function toClaudeTraceContext(actorInput: ActorExecutionInput): {
sessionId: string;
nodeId: string;
attempt: number;
depth: number;
} {
return {
sessionId: actorInput.sessionId,
nodeId: actorInput.node.id,
attempt: actorInput.attempt,
depth: actorInput.depth,
};
}
function toProviderUsageJson(usage: ProviderUsage): JsonObject {
const output: JsonObject = {};
if (typeof usage.tokenInput === "number") {
output.tokenInput = usage.tokenInput;
}
if (typeof usage.tokenOutput === "number") {
output.tokenOutput = usage.tokenOutput;
}
if (typeof usage.tokenTotal === "number") {
output.tokenTotal = usage.tokenTotal;
}
if (typeof usage.durationMs === "number") {
output.durationMs = usage.durationMs;
}
if (typeof usage.costUsd === "number") {
output.costUsd = usage.costUsd;
}
return output;
}
function buildClaudeOptions(input: { function buildClaudeOptions(input: {
runtime: ProviderRunRuntime; runtime: ProviderRunRuntime;
actorInput: ActorExecutionInput; actorInput: ActorExecutionInput;
@@ -433,6 +469,7 @@ function buildClaudeOptions(input: {
...runtime.sessionContext.runtimeInjection.env, ...runtime.sessionContext.runtimeInjection.env,
...buildClaudeAuthEnv(runtime.config.provider), ...buildClaudeAuthEnv(runtime.config.provider),
}; };
const traceContext = toClaudeTraceContext(actorInput);
return { return {
maxTurns: CLAUDE_PROVIDER_MAX_TURNS, maxTurns: CLAUDE_PROVIDER_MAX_TURNS,
@@ -449,6 +486,9 @@ function buildClaudeOptions(input: {
canUseTool: actorInput.mcp.createClaudeCanUseTool(), canUseTool: actorInput.mcp.createClaudeCanUseTool(),
cwd: runtime.sessionContext.runtimeInjection.workingDirectory, cwd: runtime.sessionContext.runtimeInjection.workingDirectory,
env: runtimeEnv, env: runtimeEnv,
...runtime.claudeObservability.toOptionOverrides({
context: traceContext,
}),
outputFormat: CLAUDE_OUTPUT_FORMAT, outputFormat: CLAUDE_OUTPUT_FORMAT,
}; };
} }
@@ -458,10 +498,19 @@ async function runClaudeTurn(input: {
actorInput: ActorExecutionInput; actorInput: ActorExecutionInput;
prompt: string; prompt: string;
}): Promise<ClaudeTurnResult> { }): Promise<ClaudeTurnResult> {
const traceContext = toClaudeTraceContext(input.actorInput);
const options = buildClaudeOptions({ const options = buildClaudeOptions({
runtime: input.runtime, runtime: input.runtime,
actorInput: input.actorInput, actorInput: input.actorInput,
}); });
input.runtime.claudeObservability.recordQueryStarted({
context: traceContext,
data: {
...(options.model ? { model: options.model } : {}),
maxTurns: options.maxTurns ?? CLAUDE_PROVIDER_MAX_TURNS,
cwd: input.runtime.sessionContext.runtimeInjection.workingDirectory,
},
});
const startedAt = Date.now(); const startedAt = Date.now();
const stream = query({ const stream = query({
@@ -472,6 +521,7 @@ async function runClaudeTurn(input: {
let resultText = ""; let resultText = "";
let structuredOutput: unknown; let structuredOutput: unknown;
let usage: ProviderUsage = {}; let usage: ProviderUsage = {};
let messageCount = 0;
const onAbort = (): void => { const onAbort = (): void => {
stream.close(); stream.close();
@@ -481,6 +531,12 @@ async function runClaudeTurn(input: {
try { try {
for await (const message of stream as AsyncIterable<SDKMessage>) { for await (const message of stream as AsyncIterable<SDKMessage>) {
messageCount += 1;
input.runtime.claudeObservability.recordMessage({
context: traceContext,
message,
});
if (message.type !== "result") { if (message.type !== "result") {
continue; continue;
} }
@@ -502,6 +558,12 @@ async function runClaudeTurn(input: {
costUsd: message.total_cost_usd, costUsd: message.total_cost_usd,
}; };
} }
} catch (error) {
input.runtime.claudeObservability.recordQueryError({
context: traceContext,
error,
});
throw error;
} finally { } finally {
input.actorInput.signal.removeEventListener("abort", onAbort); input.actorInput.signal.removeEventListener("abort", onAbort);
stream.close(); stream.close();
@@ -512,9 +574,22 @@ async function runClaudeTurn(input: {
} }
if (!resultText) { if (!resultText) {
throw new Error("Claude run completed without a final result."); const error = new Error("Claude run completed without a final result.");
input.runtime.claudeObservability.recordQueryError({
context: traceContext,
error,
});
throw error;
} }
input.runtime.claudeObservability.recordQueryCompleted({
context: traceContext,
data: {
messageCount,
usage: toProviderUsageJson(usage),
},
});
return { return {
text: resultText, text: resultText,
structuredOutput, structuredOutput,
@@ -554,19 +629,29 @@ export async function createProviderRunRuntime(input: {
initialPrompt: string; initialPrompt: string;
config: Readonly<AppConfig>; config: Readonly<AppConfig>;
projectPath: string; projectPath: string;
observabilityRootPath?: string;
}): Promise<ProviderRunRuntime> { }): Promise<ProviderRunRuntime> {
const sessionContext = await createSessionContext(input.provider, { const sessionContext = await createSessionContext(input.provider, {
prompt: input.initialPrompt, prompt: input.initialPrompt,
config: input.config, config: input.config,
workspaceRoot: input.projectPath, workspaceRoot: input.projectPath,
}); });
const claudeObservability = new ClaudeObservabilityLogger({
workspaceRoot: input.observabilityRootPath ?? input.projectPath,
config: input.config.provider.claudeObservability,
});
return { return {
provider: input.provider, provider: input.provider,
config: input.config, config: input.config,
sessionContext, sessionContext,
claudeObservability,
close: async () => { close: async () => {
try {
await sessionContext.close(); await sessionContext.close();
} finally {
await claudeObservability.close();
}
}, },
}; };
} }

View File

@@ -39,6 +39,9 @@ const dom = {
eventsLimit: document.querySelector("#events-limit"), eventsLimit: document.querySelector("#events-limit"),
eventsRefresh: document.querySelector("#events-refresh"), eventsRefresh: document.querySelector("#events-refresh"),
eventFeed: document.querySelector("#event-feed"), eventFeed: document.querySelector("#event-feed"),
claudeEventsLimit: document.querySelector("#claude-events-limit"),
claudeEventsRefresh: document.querySelector("#claude-events-refresh"),
claudeEventFeed: document.querySelector("#claude-event-feed"),
historyRefresh: document.querySelector("#history-refresh"), historyRefresh: document.querySelector("#history-refresh"),
historyBody: document.querySelector("#history-body"), historyBody: document.querySelector("#history-body"),
notificationsForm: document.querySelector("#notifications-form"), notificationsForm: document.querySelector("#notifications-form"),
@@ -147,6 +150,7 @@ const LABEL_HELP_BY_CONTROL = Object.freeze({
"session-project-path": "Absolute project path used when creating an explicit managed session.", "session-project-path": "Absolute project path used when creating an explicit managed session.",
"session-close-merge": "When enabled, close will merge the session base branch back into the project branch.", "session-close-merge": "When enabled, close will merge the session base branch back into the project branch.",
"events-limit": "Set how many recent runtime events are loaded per refresh.", "events-limit": "Set how many recent runtime events are loaded per refresh.",
"claude-events-limit": "Set how many Claude SDK trace records are loaded per refresh.",
"cfg-webhook-url": "Webhook endpoint that receives runtime event notifications.", "cfg-webhook-url": "Webhook endpoint that receives runtime event notifications.",
"cfg-webhook-severity": "Minimum severity level that triggers webhook notifications.", "cfg-webhook-severity": "Minimum severity level that triggers webhook notifications.",
"cfg-webhook-always": "Event types that should always notify, regardless of severity.", "cfg-webhook-always": "Event types that should always notify, regardless of severity.",
@@ -1493,6 +1497,43 @@ function renderEventFeed(events) {
dom.eventFeed.innerHTML = rows || '<div class="event-row"><div class="event-time">-</div><div class="event-type">-</div><div>No runtime events.</div></div>'; dom.eventFeed.innerHTML = rows || '<div class="event-row"><div class="event-time">-</div><div class="event-type">-</div><div>No runtime events.</div></div>';
} }
function toClaudeRowSeverity(event) {
const stage = String(event?.stage || "");
const type = String(event?.sdkMessageType || "");
if (stage === "query.error") {
return "critical";
}
if (stage === "query.stderr" || (type === "result" && String(event?.sdkMessageSubtype || "").startsWith("error_"))) {
return "warning";
}
return "info";
}
function renderClaudeTraceFeed(events) {
const rows = [...events]
.reverse()
.map((event) => {
const ts = new Date(event.timestamp).toLocaleTimeString();
const stage = String(event.stage || "query.message");
const sdkMessageType = String(event.sdkMessageType || "");
const sdkMessageSubtype = String(event.sdkMessageSubtype || "");
const typeLabel = sdkMessageType
? `${stage}/${sdkMessageType}${sdkMessageSubtype ? `:${sdkMessageSubtype}` : ""}`
: stage;
const message = typeof event.message === "string" ? event.message : JSON.stringify(event.message || "");
return `
<div class="event-row ${escapeHtml(toClaudeRowSeverity(event))}">
<div class="event-time">${escapeHtml(ts)}</div>
<div class="event-type">${escapeHtml(typeLabel)}</div>
<div>${escapeHtml(message)}</div>
</div>
`;
})
.join("");
dom.claudeEventFeed.innerHTML = rows || '<div class="event-row"><div class="event-time">-</div><div class="event-type">-</div><div>No Claude trace events.</div></div>';
}
async function refreshEvents() { async function refreshEvents() {
const limit = Number(dom.eventsLimit.value || "150"); const limit = Number(dom.eventsLimit.value || "150");
const params = new URLSearchParams({ const params = new URLSearchParams({
@@ -1507,6 +1548,20 @@ async function refreshEvents() {
renderEventFeed(payload.events || []); renderEventFeed(payload.events || []);
} }
async function refreshClaudeTrace() {
const limit = Number(dom.claudeEventsLimit.value || "150");
const params = new URLSearchParams({
limit: String(limit),
});
if (state.selectedSessionId) {
params.set("sessionId", state.selectedSessionId);
}
const payload = await apiRequest(`/api/claude-trace?${params.toString()}`);
renderClaudeTraceFeed(payload.events || []);
}
async function startRun(event) { async function startRun(event) {
event.preventDefault(); event.preventDefault();
@@ -1581,6 +1636,7 @@ async function startRun(event) {
dom.sessionSelect.value = run.sessionId; dom.sessionSelect.value = run.sessionId;
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
} catch (error) { } catch (error) {
showRunStatus(error instanceof Error ? error.message : String(error), true); showRunStatus(error instanceof Error ? error.message : String(error), true);
} }
@@ -1601,6 +1657,7 @@ async function cancelActiveRun() {
await loadSessions(); await loadSessions();
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
} catch (error) { } catch (error) {
showRunStatus(error instanceof Error ? error.message : String(error), true); showRunStatus(error instanceof Error ? error.message : String(error), true);
} }
@@ -1633,6 +1690,7 @@ async function createSessionFromUi() {
dom.sessionSelect.value = state.selectedSessionId; dom.sessionSelect.value = state.selectedSessionId;
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
} }
} catch (error) { } catch (error) {
showRunStatus(error instanceof Error ? error.message : String(error), true); showRunStatus(error instanceof Error ? error.message : String(error), true);
@@ -1659,6 +1717,7 @@ async function closeSelectedSessionFromUi() {
await loadSessions(); await loadSessions();
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
} catch (error) { } catch (error) {
showRunStatus(error instanceof Error ? error.message : String(error), true); showRunStatus(error instanceof Error ? error.message : String(error), true);
} }
@@ -1808,6 +1867,7 @@ function bindUiEvents() {
state.selectedSessionId = dom.sessionSelect.value; state.selectedSessionId = dom.sessionSelect.value;
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
}); });
dom.graphManifestSelect.addEventListener("change", async () => { dom.graphManifestSelect.addEventListener("change", async () => {
@@ -1827,9 +1887,14 @@ function bindUiEvents() {
await refreshEvents(); await refreshEvents();
}); });
dom.claudeEventsRefresh.addEventListener("click", async () => {
await refreshClaudeTrace();
});
dom.historyRefresh.addEventListener("click", async () => { dom.historyRefresh.addEventListener("click", async () => {
await loadSessions(); await loadSessions();
await refreshGraph(); await refreshGraph();
await refreshClaudeTrace();
}); });
dom.runForm.addEventListener("submit", startRun); dom.runForm.addEventListener("submit", startRun);
@@ -1949,6 +2014,7 @@ async function refreshAll() {
await refreshGraph(); await refreshGraph();
await refreshEvents(); await refreshEvents();
await refreshClaudeTrace();
} }
async function initialize() { async function initialize() {
@@ -1979,6 +2045,10 @@ async function initialize() {
void refreshEvents(); void refreshEvents();
}, 3000); }, 3000);
setInterval(() => {
void refreshClaudeTrace();
}, 3000);
setInterval(() => { setInterval(() => {
void refreshGraph(); void refreshGraph();
}, 7000); }, 7000);

View File

@@ -130,6 +130,24 @@
<div id="event-feed" class="event-feed"></div> <div id="event-feed" class="event-feed"></div>
</section> </section>
<section class="panel claude-panel">
<div class="panel-head">
<h2>Claude Trace</h2>
<div class="panel-actions">
<label>
Limit
<select id="claude-events-limit">
<option value="80">80</option>
<option value="150" selected>150</option>
<option value="300">300</option>
</select>
</label>
<button id="claude-events-refresh" type="button">Refresh</button>
</div>
</div>
<div id="claude-event-feed" class="event-feed claude-event-feed"></div>
</section>
<section class="panel history-panel"> <section class="panel history-panel">
<div class="panel-head"> <div class="panel-head">
<h2>Run History</h2> <h2>Run History</h2>

View File

@@ -79,7 +79,8 @@ p {
grid-template-columns: minmax(0, 2fr) minmax(280px, 1fr); grid-template-columns: minmax(0, 2fr) minmax(280px, 1fr);
grid-template-areas: grid-template-areas:
"graph side" "graph side"
"feed history" "feed claude"
"history history"
"config config"; "config config";
} }
@@ -129,6 +130,10 @@ p {
grid-area: history; grid-area: history;
} }
.claude-panel {
grid-area: claude;
}
.config-panel { .config-panel {
grid-area: config; grid-area: config;
} }
@@ -314,6 +319,14 @@ button.danger {
color: var(--critical); color: var(--critical);
} }
.claude-event-feed .event-row {
grid-template-columns: 110px 150px 1fr;
}
.claude-event-feed .event-type {
font-size: 0.7rem;
}
.history-table { .history-table {
width: 100%; width: 100%;
border-collapse: collapse; border-collapse: collapse;
@@ -485,6 +498,7 @@ button.danger {
"graph" "graph"
"side" "side"
"feed" "feed"
"claude"
"history" "history"
"config"; "config";
} }

View File

@@ -488,6 +488,7 @@ export class UiRunService {
initialPrompt: input.prompt, initialPrompt: input.prompt,
config, config,
projectPath: session?.baseWorkspacePath ?? this.workspaceRoot, projectPath: session?.baseWorkspacePath ?? this.workspaceRoot,
observabilityRootPath: this.workspaceRoot,
}); });
} }

View File

@@ -6,6 +6,7 @@ import { buildSessionGraphInsight, buildSessionSummaries } from "./session-insig
import { UiConfigStore, type LimitSettings, type RuntimeNotificationSettings, type SecurityPolicySettings } from "./config-store.js"; import { UiConfigStore, type LimitSettings, type RuntimeNotificationSettings, type SecurityPolicySettings } from "./config-store.js";
import { ManifestStore } from "./manifest-store.js"; import { ManifestStore } from "./manifest-store.js";
import { filterRuntimeEvents, readRuntimeEvents } from "./runtime-events-store.js"; import { filterRuntimeEvents, readRuntimeEvents } from "./runtime-events-store.js";
import { filterClaudeTraceEvents, readClaudeTraceEvents } from "./claude-trace-store.js";
import { parseJsonBody, sendJson, methodNotAllowed, notFound, serveStaticFile } from "./http-utils.js"; import { parseJsonBody, sendJson, methodNotAllowed, notFound, serveStaticFile } from "./http-utils.js";
import { readRunMetaBySession, UiRunService, type RunExecutionMode } from "./run-service.js"; import { readRunMetaBySession, UiRunService, type RunExecutionMode } from "./run-service.js";
import type { RunProvider } from "./provider-executor.js"; import type { RunProvider } from "./provider-executor.js";
@@ -120,11 +121,13 @@ function ensureNonEmptyString(value: unknown, field: string): string {
async function readRuntimePaths(configStore: UiConfigStore, workspaceRoot: string): Promise<{ async function readRuntimePaths(configStore: UiConfigStore, workspaceRoot: string): Promise<{
stateRoot: string; stateRoot: string;
runtimeEventLogPath: string; runtimeEventLogPath: string;
claudeTraceLogPath: string;
}> { }> {
const snapshot = await configStore.readSnapshot(); const snapshot = await configStore.readSnapshot();
return { return {
stateRoot: resolve(workspaceRoot, snapshot.paths.stateRoot), stateRoot: resolve(workspaceRoot, snapshot.paths.stateRoot),
runtimeEventLogPath: resolve(workspaceRoot, snapshot.paths.runtimeEventLogPath), runtimeEventLogPath: resolve(workspaceRoot, snapshot.paths.runtimeEventLogPath),
claudeTraceLogPath: resolve(workspaceRoot, snapshot.paths.claudeTraceLogPath),
}; };
} }
@@ -313,6 +316,27 @@ async function handleApiRequest(input: {
return true; return true;
} }
if (pathname === "/api/claude-trace") {
if (method !== "GET") {
methodNotAllowed(response);
return true;
}
const { claudeTraceLogPath } = await readRuntimePaths(configStore, workspaceRoot);
const limit = parseLimit(requestUrl.searchParams.get("limit"), 200);
const sessionId = requestUrl.searchParams.get("sessionId") ?? undefined;
const events = filterClaudeTraceEvents(await readClaudeTraceEvents(claudeTraceLogPath), {
...(sessionId ? { sessionId } : {}),
limit,
});
sendJson(response, 200, {
ok: true,
events,
});
return true;
}
if (pathname === "/api/sessions") { if (pathname === "/api/sessions") {
if (method === "POST") { if (method === "POST") {
const body = await parseJsonBody<CreateSessionRequest>(request); const body = await parseJsonBody<CreateSessionRequest>(request);

View File

@@ -0,0 +1,296 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, readFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { SDKMessage } from "@anthropic-ai/claude-agent-sdk";
import { ClaudeObservabilityLogger, summarizeClaudeMessage } from "../src/ui/claude-observability.js";
test("summarizeClaudeMessage returns compact result metadata in summary mode", () => {
const message = {
type: "result",
subtype: "success",
stop_reason: "end_turn",
num_turns: 1,
total_cost_usd: 0.0012,
usage: {
input_tokens: 120,
output_tokens: 40,
},
result: "{\"status\":\"success\"}",
duration_ms: 40,
duration_api_ms: 32,
is_error: false,
modelUsage: {},
permission_denials: [],
uuid: "uuid-1",
session_id: "sdk-session-1",
} as unknown as SDKMessage;
const summary = summarizeClaudeMessage(message, "summary");
assert.equal(summary.messageType, "result");
assert.equal(summary.messageSubtype, "success");
assert.equal(summary.sdkSessionId, "sdk-session-1");
assert.equal(summary.summary, "Claude query result success.");
assert.equal(summary.data?.numTurns, 1);
const usage = summary.data?.usage as Record<string, unknown> | undefined;
assert.equal(usage?.input_tokens, 120);
});
test("summarizeClaudeMessage redacts sensitive fields in full mode", () => {
const message = {
type: "system",
subtype: "init",
session_id: "sdk-session-2",
uuid: "uuid-2",
apiKey: "top-secret",
nested: {
authToken: "really-secret",
ok: true,
},
} as unknown as SDKMessage;
const summary = summarizeClaudeMessage(message, "full");
const payload = summary.data?.message as Record<string, unknown> | undefined;
const nested = payload?.nested as Record<string, unknown> | undefined;
assert.equal(summary.messageType, "system");
assert.equal(summary.messageSubtype, "init");
assert.equal(payload?.apiKey, "[redacted]");
assert.equal(nested?.authToken, "[redacted]");
assert.equal(nested?.ok, true);
});
test("ClaudeObservabilityLogger samples tool_progress messages for stdout", () => {
const lines: string[] = [];
const originalLog = console.log;
const originalNow = Date.now;
let now = 1000;
console.log = (line?: unknown) => {
lines.push(String(line ?? ""));
};
Date.now = () => now;
try {
const logger = new ClaudeObservabilityLogger({
workspaceRoot: process.cwd(),
config: {
mode: "stdout",
verbosity: "summary",
logPath: ".ai_ops/events/claude-trace.ndjson",
includePartialMessages: false,
debug: false,
},
});
const context = {
sessionId: "session-a",
nodeId: "node-a",
attempt: 1,
depth: 0,
};
const makeMessage = (): SDKMessage =>
({
type: "tool_progress",
tool_name: "Bash",
tool_use_id: "tool-1",
parent_tool_use_id: null,
elapsed_time_seconds: 1,
uuid: "uuid-tool",
session_id: "sdk-session-tool",
}) as unknown as SDKMessage;
logger.recordMessage({
context,
message: makeMessage(),
});
now += 300;
logger.recordMessage({
context,
message: makeMessage(),
});
now += 1200;
logger.recordMessage({
context,
message: makeMessage(),
});
assert.equal(lines.length, 2);
assert.match(lines[0] ?? "", /^\[claude-trace\] /);
assert.match(lines[1] ?? "", /"suppressedSinceLastEmit":1/);
} finally {
console.log = originalLog;
Date.now = originalNow;
}
});
test("ClaudeObservabilityLogger keeps assistant/user message records in file output", async () => {
const workspace = await mkdtemp(join(tmpdir(), "claude-obsv-test-"));
const logPath = ".ai_ops/events/claude-trace.ndjson";
const logger = new ClaudeObservabilityLogger({
workspaceRoot: workspace,
config: {
mode: "file",
verbosity: "summary",
logPath,
includePartialMessages: false,
debug: false,
},
});
const context = {
sessionId: "session-file",
nodeId: "node-file",
attempt: 1,
depth: 0,
};
logger.recordQueryStarted({
context,
});
logger.recordMessage({
context,
message: {
type: "assistant",
uuid: "assistant-1",
session_id: "sdk-file-1",
parent_tool_use_id: null,
message: {} as never,
} as unknown as SDKMessage,
});
logger.recordMessage({
context,
message: {
type: "user",
uuid: "user-1",
session_id: "sdk-file-1",
parent_tool_use_id: null,
message: {} as never,
} as unknown as SDKMessage,
});
logger.recordMessage({
context,
message: {
type: "result",
subtype: "success",
stop_reason: "end_turn",
num_turns: 1,
total_cost_usd: 0.0012,
usage: {
input_tokens: 100,
output_tokens: 20,
},
result: "{}",
duration_ms: 10,
duration_api_ms: 9,
is_error: false,
modelUsage: {},
permission_denials: [],
uuid: "result-1",
session_id: "sdk-file-1",
} as unknown as SDKMessage,
});
logger.recordQueryCompleted({
context,
});
await logger.close();
const filePath = join(workspace, logPath);
const content = await readFile(filePath, "utf8");
const lines = content.split(/\r?\n/).filter((line) => line.trim().length > 0);
const records = lines.map((line) => JSON.parse(line) as Record<string, unknown>);
const messageTypes = records
.map((record) => record.sdkMessageType)
.filter((value) => typeof value === "string");
assert.equal(messageTypes.includes("assistant"), true);
assert.equal(messageTypes.includes("user"), true);
assert.equal(messageTypes.includes("result"), true);
});
test("summarizeClaudeMessage maps task_notification system subtype", () => {
const message = {
type: "system",
subtype: "task_notification",
task_id: "task-1",
status: "completed",
output_file: "/tmp/out.txt",
summary: "Task complete",
uuid: "uuid-task",
session_id: "sdk-session-task",
} as unknown as SDKMessage;
const summary = summarizeClaudeMessage(message, "summary");
assert.equal(summary.messageType, "system");
assert.equal(summary.messageSubtype, "task_notification");
assert.equal(summary.summary, "Task notification: completed.");
assert.equal(summary.data?.taskId, "task-1");
});
test("ClaudeObservabilityLogger honors includePartialMessages for stream events", () => {
const lines: string[] = [];
const originalLog = console.log;
console.log = (line?: unknown) => {
lines.push(String(line ?? ""));
};
try {
const context = {
sessionId: "session-stream",
nodeId: "node-stream",
attempt: 1,
depth: 0,
};
const streamMessage = {
type: "stream_event",
event: {
type: "content_block_delta",
},
parent_tool_use_id: null,
uuid: "stream-1",
session_id: "sdk-session-stream",
} as unknown as SDKMessage;
const withoutPartial = new ClaudeObservabilityLogger({
workspaceRoot: process.cwd(),
config: {
mode: "stdout",
verbosity: "summary",
logPath: ".ai_ops/events/claude-trace.ndjson",
includePartialMessages: false,
debug: false,
},
});
withoutPartial.recordMessage({
context,
message: streamMessage,
});
const withPartial = new ClaudeObservabilityLogger({
workspaceRoot: process.cwd(),
config: {
mode: "stdout",
verbosity: "summary",
logPath: ".ai_ops/events/claude-trace.ndjson",
includePartialMessages: true,
debug: false,
},
});
withPartial.recordMessage({
context,
message: streamMessage,
});
assert.equal(lines.length, 1);
assert.match(lines[0] ?? "", /\"sdkMessageType\":\"stream_event\"/);
} finally {
console.log = originalLog;
}
});

View File

@@ -0,0 +1,42 @@
import test from "node:test";
import assert from "node:assert/strict";
import { mkdtemp, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { filterClaudeTraceEvents, readClaudeTraceEvents } from "../src/ui/claude-trace-store.js";
test("readClaudeTraceEvents parses and sorts ndjson records", async () => {
const workspace = await mkdtemp(join(tmpdir(), "claude-trace-store-"));
const logPath = join(workspace, "claude-trace.ndjson");
await writeFile(
logPath,
[
'{"timestamp":"2026-02-24T17:27:05.000Z","message":"later","sessionId":"s1"}',
'not-json',
'{"timestamp":"2026-02-24T17:26:00.000Z","message":"earlier","sessionId":"s1"}',
'{"message":"missing timestamp"}',
].join("\n"),
"utf8",
);
const events = await readClaudeTraceEvents(logPath);
assert.equal(events.length, 2);
assert.equal(events[0]?.message, "earlier");
assert.equal(events[1]?.message, "later");
});
test("filterClaudeTraceEvents filters by session and limit", () => {
const events = [
{ timestamp: "2026-02-24T17:00:00.000Z", message: "a", sessionId: "s1" },
{ timestamp: "2026-02-24T17:01:00.000Z", message: "b", sessionId: "s2" },
{ timestamp: "2026-02-24T17:02:00.000Z", message: "c", sessionId: "s1" },
];
const filtered = filterClaudeTraceEvents(events, {
sessionId: "s1",
limit: 1,
});
assert.equal(filtered.length, 1);
assert.equal(filtered[0]?.message, "c");
});

View File

@@ -25,6 +25,11 @@ test("loads defaults and freezes config", () => {
"session.failed", "session.failed",
]); ]);
assert.equal(config.provider.openAiAuthMode, "auto"); assert.equal(config.provider.openAiAuthMode, "auto");
assert.equal(config.provider.claudeObservability.mode, "off");
assert.equal(config.provider.claudeObservability.verbosity, "summary");
assert.equal(config.provider.claudeObservability.logPath, ".ai_ops/events/claude-trace.ndjson");
assert.equal(config.provider.claudeObservability.includePartialMessages, false);
assert.equal(config.provider.claudeObservability.debug, false);
assert.equal(Object.isFrozen(config), true); assert.equal(Object.isFrozen(config), true);
assert.equal(Object.isFrozen(config.orchestration), true); assert.equal(Object.isFrozen(config.orchestration), true);
}); });
@@ -57,6 +62,38 @@ test("validates runtime discord severity mode", () => {
); );
}); });
test("validates claude observability mode", () => {
assert.throws(
() => loadConfig({ CLAUDE_OBSERVABILITY_MODE: "stream" }),
/CLAUDE_OBSERVABILITY_MODE must be one of/,
);
});
test("validates claude observability verbosity", () => {
assert.throws(
() => loadConfig({ CLAUDE_OBSERVABILITY_VERBOSITY: "verbose" }),
/CLAUDE_OBSERVABILITY_VERBOSITY must be one of/,
);
});
test("loads claude observability settings", () => {
const config = loadConfig({
CLAUDE_OBSERVABILITY_MODE: "both",
CLAUDE_OBSERVABILITY_VERBOSITY: "full",
CLAUDE_OBSERVABILITY_LOG_PATH: ".ai_ops/debug/claude.ndjson",
CLAUDE_OBSERVABILITY_INCLUDE_PARTIAL: "true",
CLAUDE_OBSERVABILITY_DEBUG: "true",
CLAUDE_OBSERVABILITY_DEBUG_LOG_PATH: ".ai_ops/debug/claude-sdk.log",
});
assert.equal(config.provider.claudeObservability.mode, "both");
assert.equal(config.provider.claudeObservability.verbosity, "full");
assert.equal(config.provider.claudeObservability.logPath, ".ai_ops/debug/claude.ndjson");
assert.equal(config.provider.claudeObservability.includePartialMessages, true);
assert.equal(config.provider.claudeObservability.debug, true);
assert.equal(config.provider.claudeObservability.debugLogPath, ".ai_ops/debug/claude-sdk.log");
});
test("prefers CLAUDE_CODE_OAUTH_TOKEN over ANTHROPIC_API_KEY", () => { test("prefers CLAUDE_CODE_OAUTH_TOKEN over ANTHROPIC_API_KEY", () => {
const config = loadConfig({ const config = loadConfig({
CLAUDE_CODE_OAUTH_TOKEN: "oauth-token", CLAUDE_CODE_OAUTH_TOKEN: "oauth-token",

View File

@@ -155,3 +155,41 @@ test("secure executor runs with explicit env policy", async () => {
assert.equal(result.stdout, "ok|\n"); assert.equal(result.stdout, "ok|\n");
assert.equal(streamedStdout, result.stdout); assert.equal(streamedStdout, result.stdout);
}); });
test("rules engine carries session context in tool audit events", () => {
const events: Array<Record<string, unknown>> = [];
const rules = new SecurityRulesEngine(
{
allowedBinaries: ["git"],
worktreeRoot: "/tmp",
protectedPaths: [],
requireCwdWithinWorktree: true,
rejectRelativePathTraversal: true,
enforcePathBoundaryOnArguments: true,
allowedEnvAssignments: [],
blockedEnvAssignments: [],
},
(event) => {
events.push(event as unknown as Record<string, unknown>);
},
);
rules.assertToolInvocationAllowed({
tool: "git",
toolClearance: {
allowlist: ["git"],
banlist: [],
},
context: {
sessionId: "session-ctx",
nodeId: "node-ctx",
attempt: 2,
},
});
const allowedEvent = events.find((event) => event.type === "tool.invocation_allowed");
assert.ok(allowedEvent);
assert.equal(allowedEvent.sessionId, "session-ctx");
assert.equal(allowedEvent.nodeId, "node-ctx");
assert.equal(allowedEvent.attempt, 2);
});

View File

@@ -1,7 +1,7 @@
import test from "node:test"; import test from "node:test";
import assert from "node:assert/strict"; import assert from "node:assert/strict";
import { execFile } from "node:child_process"; import { execFile } from "node:child_process";
import { mkdtemp, mkdir, readFile, writeFile, stat } from "node:fs/promises"; import { mkdtemp, mkdir, readFile, rm, stat, writeFile } from "node:fs/promises";
import { tmpdir } from "node:os"; import { tmpdir } from "node:os";
import { resolve } from "node:path"; import { resolve } from "node:path";
import { promisify } from "node:util"; import { promisify } from "node:util";
@@ -177,3 +177,54 @@ test("session worktree manager returns conflict outcome instead of throwing", as
assert.equal(mergeOutcome.worktreePath, taskWorktreePath); assert.equal(mergeOutcome.worktreePath, taskWorktreePath);
assert.ok(mergeOutcome.conflictFiles.includes("README.md")); assert.ok(mergeOutcome.conflictFiles.includes("README.md"));
}); });
test("session worktree manager recreates a task worktree after stale metadata prune", async () => {
const root = await mkdtemp(resolve(tmpdir(), "ai-ops-session-worktree-prune-"));
const projectPath = resolve(root, "project");
const worktreeRoot = resolve(root, "worktrees");
await mkdir(projectPath, { recursive: true });
await git(["init", projectPath]);
await git(["-C", projectPath, "config", "user.name", "AI Ops"]);
await git(["-C", projectPath, "config", "user.email", "ai-ops@example.local"]);
await writeFile(resolve(projectPath, "README.md"), "# project\n", "utf8");
await git(["-C", projectPath, "add", "README.md"]);
await git(["-C", projectPath, "commit", "-m", "initial commit"]);
const manager = new SessionWorktreeManager({
worktreeRoot,
baseRef: "HEAD",
});
const sessionId = "session-prune-1";
const taskId = "task-prune-1";
const baseWorkspacePath = manager.resolveBaseWorkspacePath(sessionId);
await manager.initializeSessionBaseWorkspace({
sessionId,
projectPath,
baseWorkspacePath,
});
const initialTaskWorktreePath = (
await manager.ensureTaskWorktree({
sessionId,
taskId,
baseWorkspacePath,
})
).taskWorktreePath;
await rm(initialTaskWorktreePath, { recursive: true, force: true });
const recreatedTaskWorktreePath = (
await manager.ensureTaskWorktree({
sessionId,
taskId,
baseWorkspacePath,
})
).taskWorktreePath;
assert.equal(recreatedTaskWorktreePath, initialTaskWorktreePath);
const stats = await stat(recreatedTaskWorktreePath);
assert.equal(stats.isDirectory(), true);
});