Writing a Framework Adapter

An adapter translates data your provider or framework already produces into an Attest Trace. Write one when a built-in adapter does not cover your integration. You can also use TraceBuilder directly without an adapter.

Choose a Base Class

Both classes are exported from attest and attest.adapters:

• BaseProviderAdapter builds a trace for a single provider response. Implement its four required extraction hooks; the base class handles trace construction.
• BaseAdapter provides _create_builder(), _now_ms(), and _resolve_timestamps(started_at_ms, ended_at_ms). Use it for framework callbacks or multi-step runs that need their own capture flow.

Adapters are plain Python classes. There is no adapter registration step: import and construct your adapter where you capture the response. Assertion plugins use a separate registry; see Writing a Plugin.

Install

In an existing uv project:

uv add "attest-ai>=1.1.0" pytest

The following example uses a saved response dictionary, so it needs no provider SDK, credentials, or paid calls.

Implement a Provider Adapter

Create acme_adapter.py. Acme is an example response format, not an external SDK:

from typing import Any

from attest import BaseProviderAdapter


class AcmeAdapter(BaseProviderAdapter):
    def _extract_completion(self, response: Any) -> str:
        return response.get("text", "")

    def _extract_model(self, response: Any, **metadata: Any) -> str | None:
        return metadata.get("model") or response.get("model")

    def _extract_total_tokens(self, response: Any) -> int | None:
        return response.get("usage", {}).get("total_tokens")

    def _extract_tool_calls(self, response: Any) -> list[dict[str, Any]]:
        return [
            {"name": call["name"], "args": call.get("arguments")}
            for call in response.get("tool_calls", [])
        ]

These are the four required hooks. Return None for missing usage or model information and an empty list when there are no tool calls. Each tool-call dictionary requires name and may contain args.

The inherited method has this signature:

trace_from_response(
    response: Any,
    input_messages: list[dict[str, Any]] | None = None,
    started_at_ms: int | None = None,
    ended_at_ms: int | None = None,
    **metadata: Any,
) -> Trace

This is a signature reference, not a standalone function definition. The method builds one llm_call step named completion, then one tool_call step per extracted call. The default trace output is {"message": completion_text}. Requested tool calls are not proof of execution: the provider base class does not run tools or record their returned results.

input_messages becomes trace.input["messages"] when nonempty. Trace metadata records extracted total_tokens and model, caller-supplied cost_usd, and latency. Explicit latency_ms takes precedence; otherwise latency is derived only when both caller timestamps are present. Step timestamps fall back to the current time, but that fallback is not a measurement of the original provider request.

Test Capture and Evaluation

Create test_acme_adapter.py:

import pytest

from attest import AgentResult, expect
from acme_adapter import AcmeAdapter


@pytest.fixture
def response():
    return {
        "model": "acme-large",
        "text": "Refund processed.",
        "usage": {"total_tokens": 150},
        "tool_calls": [
            {"name": "process_refund", "arguments": {"order_id": "ORD-1"}},
        ],
    }


def test_capture(response):
    adapter = AcmeAdapter(agent_id="support-agent")
    trace = adapter.trace_from_response(
        response,
        input_messages=[{"role": "user", "content": "Refund ORD-1"}],
        started_at_ms=1000,
        ended_at_ms=1200,
        cost_usd=0.005,
    )
    assert trace.agent_id == "support-agent"
    assert trace.output == {"message": "Refund processed."}
    assert trace.metadata.total_tokens == 150
    assert trace.metadata.cost_usd == 0.005
    assert trace.metadata.latency_ms == 200
    tools = [step for step in trace.steps if step.type == "tool_call"]
    assert [step.name for step in tools] == ["process_refund"]
    assert tools[0].args == {"order_id": "ORD-1"}


def test_evaluate(response, attest):
    trace = AcmeAdapter().trace_from_response(response, cost_usd=0.005)
    result = AgentResult(trace=trace)
    verdict = attest.evaluate(
        expect(result)
        .output_contains("Refund")
        .required_tools(["process_refund"])
        .cost_under(0.10)
    )
    assert verdict.passed


def test_missing_optional_data():
    trace = AcmeAdapter().trace_from_response({"text": "Hello."})
    assert trace.metadata.total_tokens is None
    assert [step for step in trace.steps if step.type == "tool_call"] == []

Run the tests:

uv run pytest test_acme_adapter.py

AgentResult(trace=trace) wraps captured data; attest.evaluate() returns the evaluated result. expect(trace) is also supported and wraps the trace automatically. Always evaluate the chain: leaving assertions unevaluated fails the test in 1.1.0.

Optional Provider Hooks

Override these only when your provider needs different handling. The signatures below include self and use Any from typing:

Hook	Default
_extract_input(self, input_messages: list[dict[str, Any]] | None, **metadata: Any) -> dict[str, Any] | None	Nonempty messages become {"messages": input_messages}
_build_output(self, response: Any, completion_text: str, **metadata: Any) -> dict[str, Any]	{"message": completion_text}
_extract_token_split(self, response: Any) -> tuple[int | None, int | None]	(None, None)
_extract_refusal(self, response: Any) -> str | None	None

Token splits are stored as input_tokens and output_tokens in the LLM step’s result. A nonempty refusal is stored as refusal in that result; if your provider treats a refusal as the visible answer, your completion hook should return that text as well. The built-in OpenAI adapter demonstrates this for Chat Completions and Responses API objects.

For structured output, extend the complete adapter rather than replacing it with an incomplete subclass:

class StructuredAcmeAdapter(AcmeAdapter):
    def _build_output(
        self, response: Any, completion_text: str, **metadata: Any
    ) -> dict[str, Any]:
        return {
            "message": completion_text,
            "structured": response.get("structured", {}),
        }

Place this class in acme_adapter.py after AcmeAdapter.

Framework Runs and Callbacks

For a framework that emits several calls, use BaseAdapter and capture actual events instead of inventing steps from its configuration. _create_builder() preserves the adapter’s agent_id. Record call arguments, results, and timestamps in framework hooks, then set the final output and call build() once.

The built-in LangChain adapter illustrates a callback-based flow. With your agent_executor already configured:

from attest import LangChainAdapter

adapter = LangChainAdapter(agent_id="support-agent")
with adapter.capture() as handler:
    agent_executor.invoke(
        {"input": "Refund ORD-1"}, config={"callbacks": [handler]}
    )
trace = adapter.trace

Install its extra with uv add "attest-ai[langchain]>=1.1.0". capture() yields a LangChainCallbackHandler and builds the trace on successful exit. The handler records callbacks that actually occur; framework configuration alone does not establish that a tool or agent ran. In particular, the CrewAI adapter produces summary-only traces without steps.

For nested agents, see the Multi-Agent guide rather than flattening child traces into a single provider response.

Timing, Costs, and Errors

Measure timestamps around the real provider request, not around conversion of an already-received response. Supply cost_usd from your billing calculation when you need cost assertions: the base class does not infer currency costs from token counts. Match your provider’s rate units when calculating costs.

Let provider and extraction errors propagate so the test reports the failed capture. Test saved responses for missing usage, empty output, refusals, structured output, and tool arguments before adding live-provider tests.

Related

• Adapters Reference — Built-in adapters and maturity
• Expect DSL Reference — Assertion methods
• Framework Adapters Guide — Adapter capture flows