JSON-RPC Protocol

The SDK sends traces and assertions to a local engine process and receives verdicts. This reference describes the 1.1.0 wire format; normal tests should use the Python or TypeScript SDK rather than construct requests themselves.

Transport and lifecycle

The protocol is JSON-RPC 2.0 over UTF-8 NDJSON on stdin/stdout: one JSON object followed by a newline per message. Engine logs go to stderr, not stdout. The stdio engine processes one request at a time; parallel embedding and judge work happens inside a batch.

Initialize once, evaluate or query as needed, then shut down. Requests carry jsonrpc, id, method, and method-specific params. In 1.1.0 the engine echoes numeric, string, or null IDs exactly. A valid request without an id is a notification: it executes without a reply. The engine emits no unsolicited drift notifications.

A minimal handshake on the wire is:

{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"sdk_name":"attest-python","sdk_version":"1.1.0","protocol_version":1,"required_capabilities":["layers_1_4"],"preferred_encoding":"json"}}

Success responses contain result; failures contain error with code, message, and optional data (error_type, retryable, detail). SDK callers correlate replies by ID, not arrival order. SDKs reject evaluation replies whose assertion IDs differ from the request. A request timeout restarts the engine and fails pending requests; it does not turn a late reply into another test’s verdict.

initialize

Parameter	Type	Meaning
sdk_name	string	Client identifier
sdk_version	string	Client version, for example 1.1.0
protocol_version	integer	Must be 1
required_capabilities	string[]	Capabilities the client needs
preferred_encoding	string	Use json; the engine returns JSON

Result field	Meaning
engine_version	Binary version (1.1.0 for this release)
protocol_version	1
capabilities	Runtime capabilities, dependent on configured providers and simulation
missing	Requested capabilities not available
compatible	Whether missing is empty
encoding	json
max_concurrent_requests	1 in stdio mode
max_trace_size_bytes	10485760 (10 MiB)
max_steps_per_trace	10000

Do not treat the capabilities list as a fixed list shared by all installations. A protocol-version mismatch returns a session error. Reinitializing the same session also returns a session error.

evaluate_batch

Parameters are trace, assertions, optional budget, and optional context. Each assertion has assertion_id, type, spec, and optional request_id and scope_id. SDKs assign stable scheme-2 assertion IDs; direct protocol clients must supply IDs and correlate results by them, not array position.

This example checks a recorded cost without making a model call:

{
  "jsonrpc": "2.0",
  "id": 2,
  "method": "evaluate_batch",
  "params": {
    "trace": {
      "schema_version": 1,
      "trace_id": "cost-example",
      "agent_id": "example",
      "input": {},
      "steps": [],
      "output": {"message": "Done"},
      "metadata": {"cost_usd": 0.001}
    },
    "assertions": [{
      "assertion_id": "cost-check",
      "type": "constraint",
      "spec": {"field": "metadata.cost_usd", "operator": "lte", "value": 0.01}
    }]
  }
}

budget can contain max_judge_calls (integer) and max_cost_usd (number). Omitted limits are unlimited at batch level; session limits still apply. These cap Attest-initiated model calls, not the agent cost recorded in the trace. context can contain test_id, file, and line; the engine uses it for assertion-spec recording when ATTEST_CONTRACTS_DIR is set.

Batch result

Field	Meaning
results	One verdict row per assertion, including skipped evaluation; correlate by assertion_id
total_cost	Sum of assertion-result costs; not a complete model-spend ledger
total_duration_ms	Sum of assertion evaluation durations, not batch wall-clock time
simulated	false for engine-produced results, including engine simulation mode
usage	Attest-initiated model usage for the batch
budget_exceeded	Optional flag when the session soft-failure budget was exceeded; all rows are still returned

usage contains model_calls, judge_calls, input_tokens, output_tokens, cached_tokens, cost_usd, retry_cost_usd, unpriced_calls, and budget_refused. Use it to account for model calls, including retries and embedding calls. Nonzero unpriced_calls means reported USD spend is incomplete. Neither this usage nor assertion evaluation cost is the agent’s own metadata.cost_usd.

Assertion results

Required row fields are assertion_id, status, score, explanation, cost, and duration_ms. Scores are evaluation scores, not statistical confidence. Status is one of:

• pass: the assertion met its contract.
• hard_fail: the assertion failed.
• soft_fail: a soft assertion failed; policy determines its CI consequence.
• not_evaluated: no verdict was produced; reason explains why. A zero score here is not a failure score.

Reasons include provider_not_configured, provider_unavailable, provider_auth, judge_malformed, judge_abstained, simulation, missing_evidence, short_circuited, and budget_exhausted. SDK replay can additionally report replay_miss.

Optional diagnostic fields include request_id, scope_id, spec_hash, threshold_source, layer, type, trace_node_path, expected, actual, suggested_action, judge_metadata, drift, failure_class, and trial. Omitted optional fields are not necessarily serialized as null. threshold_source distinguishes static, dynamic, and dynamic_unavailable (static fallback after a history lookup failure).

Assertion types

Type	Layer	Important spec fields
schema	1	target, schema
constraint	2	field, operator, value or bounds
trace	3	check and check-specific tool/argument fields
trace_tree	3	check and check-specific agent/tree fields
effect	3	check, world, resource, optional where, check-specific bounds or relations
content	4	target, check, value or values, optional case_sensitive, soft
embedding	5	target, reference, threshold, optional soft
llm_judge	6	target, criteria, rubric, threshold, optional model, soft
plugin	Plugins	SDK-computed plugin result lookup

The public model is six evaluated layers plus plugins. Trace-tree checks are Layer 3 in 1.1.0. See Assertion Pipeline for evaluation order, relational effects, and scoring.

submit_plugin_result

Submit the SDK-computed result before evaluating the corresponding plugin assertion:

{"jsonrpc":"2.0","id":3,"method":"submit_plugin_result","params":{"trace_id":"cost-example","plugin_name":"compliance","assertion_id":"plugin-check","result":{"status":"pass","score":1.0,"explanation":"Rules satisfied"}}}

result contains status, score, explanation, and optional metadata. It is not a full assertion-result row: do not send nested assertion_id, cost, or duration_ms as result fields. A valid submission returns {"accepted":true} and is stored by trace ID, assertion ID, and plugin name. It need not match a pending assertion. Submission alone neither increments evaluated-assertion statistics nor records a history verdict; evaluate_batch does that when it reports the result.

validate_trace_tree

Parameters: {"trace": <trace>}. The result contains valid, optional errors, depth, agent_count, agent_ids, aggregate_tokens, aggregate_cost_usd, and aggregate_latency_ms. This inspects the nested trace tree without requiring an assertion batch.

query_drift

Read-only query of the newest recorded assertion run against its historical baseline. Parameters:

Parameter	Meaning
assertion_id	History key from the assertion report or runner scope
static_threshold	Optional number in [0, 1]; omitted/null means 0.8; explicit zero is honored
window_size	Accepted but ignored; the window is fixed at 50

The result contains report and the resolved static_threshold. report contains assertion_id, mean, stddev, count, latest_score, deviation (latest minus mean), and status (ok or drift_detected).

The baseline uses up to 50 passing and soft-failing runs before the newest run, ordered by insertion. It needs at least 10 eligible prior runs. The bar is max(static_threshold, mean - 2 * max(stddev, 0.02)). With an active baseline, a latest score below the bar produces drift_detected; otherwise the status is ok. Hard failures do not enter the baseline. An unavailable history store returns an engine error. Drift from an evaluated dynamic assertion is also carried in results[].drift, not an unsolicited notification.

generate_user_message

Available only when a judge provider is configured. This method uses that provider to generate a user message; it is not the provider-free assertion simulation mode.

Parameters are persona (name, system_prompt, style, temperature, optional max_tokens), conversation_history (objects with role and content), and optional fault_config (error_rate, latency_jitter_ms, content_corruption, timeout_after_ms). The result is {"message":"..."}. Model-call budgets apply.

shutdown

Send params: {} after initialization. The result contains sessions_completed and assertions_evaluated; the session enters shutdown. Do not assume fixed SDK signal timings as part of the wire contract.

Errors

Code	Meaning
1001	Invalid trace
1002	Assertion/parameter error
2001	Provider error
3001	Engine error
3002	Timeout
3003	Session-state error
-32700	JSON parse error
-32600	Invalid JSON-RPC request
-32601	Method not found
-32602	Invalid parameters
-32603	Internal JSON-RPC error

An invalid assertion spec or unknown assertion type can instead produce a hard_fail row within a successful batch response. Provider failures in embedding/judge assertions normally produce not_evaluated, not an RPC error or a passing verdict.

Simulation and CI policy

ATTEST_SIMULATION=1 still runs the real engine and deterministic layers. Embedding and judge assertions return not_evaluated with reason simulation unless an earlier deterministic failure already short-circuited them. It does not fabricate passing assertions or prevent application code from calling a provider.

A judge abstention, malformed reply, or rejected credential never becomes a passing score. The default policy warns on not_evaluated; set not_evaluated: block when missing verdicts must block CI. An unevaluated expect() chain is a separate SDK error: pytest and Vitest fail a test that forgets to evaluate it, including assertions appended after evaluation.