Migrating from DeepEval

Move a test case at a time, keeping the same inputs and expected behavior. Attest 1.1.0 runs assertions inside pytest or Vitest; its judges and embedding scores are not drop-in replacements for another tool’s metrics. Recheck thresholds against labeled examples rather than copying score cutoffs.

Start with a deterministic requirement

uv add "attest-ai>=1.1.0" pytest

This complete pytest example checks a known phrase without a provider call:

from attest import TraceBuilder, expect


def test_answer_mentions_scattering(attest):
    trace = TraceBuilder("science").set_output(message="The sky looks blue because of scattering.").build()
    attest.evaluate(expect(trace).output_contains("scattering"))

Save it as test_science.py and run uv run pytest. In your application, replace the sample trace with one from your agent or adapter.

Translate the intent, not the metric name

Requirement	Attest check	Limit
Required phrase or pattern	output_contains, output_matches_regex	Text matching does not establish factual correctness.
Structured result	output_matches_schema	Checks the output’s structured field against JSON Schema.
Similarity to a reference	output_similar_to	Semantic similarity is not answer relevance or evidence-grounded faithfulness.
Subjective criterion	passes_judge	Choose a rubric and calibrate the configured provider/model.
Tool behavior or external effects	Behavior and world-effect contracts	Requires traces and world-state evidence, not only a reply string.

Preserve retrieval context and other evidence needed by your criteria. A generic judge prompt is not an automatic replacement for a retrieval-aware metric. See judges for provider setup, costs and calibration.

Always evaluate the chain: building expect(...) alone checks nothing and fails under the 1.1.0 pytest guard. Missing measurements are not_evaluated; they are not passes. Review verdicts and choose a policy, including not_evaluated: block if required.

Use the Python setup or TypeScript setup for project integration. Keep existing evaluations alongside the migrated checks until you understand their disagreements.