Skip to content

TypeScript parity

Every capability the Python SDK exposes has a TypeScript counterpart, or a documented exception. This page is generated from the public surface of both SDKs by the parity test (sdks/python/tests/test_parity.py); do not edit it by hand. Regenerate it with ATTEST_UPDATE_PARITY=1 after changing either SDK’s surface.

Python names map to TypeScript names by snake_case to camelCase, plus an explicit alias table. Capabilities that decide a test outcome (simulation scenarios, personas and fault injection, the plugin runner and its discovery, init, capture, replay and world helpers, trials) are never exceptions. Commands other than init are forwarded to the engine by both SDKs, so they are identical.

Exceptions use one of three reasons: upstream_python_only (the framework has no TypeScript release), otel_capture (TypeScript users capture the framework through OpenTelemetry) or language_idiom (the TypeScript way is named in the note).

PythonReasonTypeScript way
CrewAIAdapterupstream_python_onlyCrewAI has no TypeScript release.
GoogleADKAdapterotel_captureCapture Google ADK runs through its OpenTelemetry spans with attest capture.
LlamaIndexInstrumentationHandlerotel_captureCapture LlamaIndex runs through its OpenTelemetry spans with attest capture.
BaseProviderAdapterlanguage_idiomTypeScript adapters implement the TraceAdapter interface directly; there is no template-method base class.
evaluate_asynclanguage_idiomevaluate is already async in TypeScript.
inspect_asynclanguage_idiominspect is already async in TypeScript.
integrationlanguage_idiomVitest splits integration tests with a separate include glob or test:integration script.
PythonTypeScriptStatusNote
AnthropicAdapterAnthropicAdaptermatch-
CrewAIAdapter-exception (upstream_python_only)CrewAI has no TypeScript release.
GeminiAdapterGeminiAdaptermatch-
GoogleADKAdapter-exception (otel_capture)Capture Google ADK runs through its OpenTelemetry spans with attest capture.
LangChainAdapterLangChainAdaptermatch-
LangChainCallbackHandlerLangChainAdapteralias-
LlamaIndexInstrumentationHandler-exception (otel_capture)Capture LlamaIndex runs through its OpenTelemetry spans with attest capture.
ManualAdapterManualAdaptermatch-
OTelAdapterOTelAdaptermatch-
OllamaAdapterOllamaAdaptermatch-
OpenAIAdapterOpenAIAdaptermatch-
PythonTypeScriptStatusNote
AgentAgentmatch-
AgentResultAgentResultmatch-
AlertDispatcherAlertDispatchermatch-
AssertionAssertionmatch-
AssertionResultAssertionResultmatch-
BaseAdapterTraceAdapteralias-
BaseProviderAdapter-exception (language_idiom)TypeScript adapters implement the TraceAdapter interface directly; there is no template-method base class.
BiasProbeBiasProbematch-
CaptureErrorCaptureErrormatch-
ContinuousEvalRunnerContinuousEvalRunnermatch-
ConversationMessageConversationMessagematch-
DriftReportDriftReportmatch-
ENGINE_VERSIONENGINE_VERSIONmatch-
EgressBlockedErrorEgressBlockedErrormatch-
EngineTimeoutErrorEngineTimeoutErrormatch-
ExpectChainExpectChainmatch-
FixtureFixturematch-
JudgeAgreementJudgeAgreementmatch-
JudgeMetadataJudgeMetadatamatch-
REASON_REPLAY_MISSREASON_REPLAY_MISSmatch-
RelatedRelatedmatch-
ReplayErrorReplayErrormatch-
ReplayMissErrorReplayMissErrormatch-
ReplayNotEvaluatedErrorReplayNotEvaluatedErrormatch-
ReplaySessionReplaySessionmatch-
ReplaySummaryReplaySummarymatch-
SamplerSamplermatch-
SimulateFaultConfigSimulateFaultConfigmatch-
SimulatePersonaSimulatePersonamatch-
StepStepmatch-
TIER_1TIER_1match-
TIER_2TIER_2match-
TIER_3TIER_3match-
TraceTracematch-
TraceBuilderTraceBuildermatch-
TraceMetadataTraceMetadatamatch-
TraceTreeTraceTreematch-
UsageUsagematch-
WorldErrorWorldErrormatch-
WorldSessionWorldSessionmatch-
agentagentmatch-
configconfigmatch-
delegatedelegatematch-
egress_block_requestedegressBlockRequestedmatch-
expectattestExpectalias-
first_missfirstMissmatch-
install_egress_guardinstallEgressGuardmatch-
is_provider_hostisProviderHostmatch-
live_lineliveLinematch-
load_fixtureloadFixturematch-
recordrecordmatch-
relatedrelatedmatch-
resolve_moderesolveModematch-
start_replaystartReplaymatch-
start_worldstartWorldmatch-
tiertiermatch-
uninstall_egress_guarduninstallEgressGuardmatch-
PythonTypeScriptStatusNote
agent_calledagentCalledmatch-
agent_ordered_beforeagentOrderedBeforematch-
agent_output_containsagentOutputContainsmatch-
agent_wall_time_underagentWallTimeUndermatch-
agents_overlapagentsOverlapmatch-
aggregate_cost_underaggregateCostUndermatch-
aggregate_latency_underaggregateLatencyUndermatch-
aggregate_tokens_underaggregateTokensUndermatch-
all_tools_calledallToolsCalledmatch-
assertionsassertionsmatch-
constraintconstraintmatch-
content_containscontentContainsmatch-
cost_undercostUndermatch-
cross_agent_data_flowcrossAgentDataFlowmatch-
delegation_depthdelegationDepthmatch-
evaluatedevaluatedmatch-
follows_transitionsfollowsTransitionsmatch-
forbidden_toolsforbiddenToolsmatch-
latency_underlatencyUndermatch-
max_tool_failure_streakmaxToolFailureStreakmatch-
no_duplicate_toolsnoDuplicateToolsmatch-
no_tool_cyclesnoToolCyclesmatch-
no_tool_loopsnoToolLoopsmatch-
ordered_agentsorderedAgentsmatch-
output_containsoutputContainsmatch-
output_field_matches_schemaoutputFieldMatchesSchemamatch-
output_forbidsoutputForbidsmatch-
output_has_all_keywordsoutputHasAllKeywordsmatch-
output_has_any_keywordoutputHasAnyKeywordmatch-
output_matches_regexoutputMatchesRegexmatch-
output_matches_schemaoutputMatchesSchemamatch-
output_not_containsoutputNotContainsmatch-
output_similar_tooutputSimilarTomatch-
passes_judgepassesJudgematch-
pluginpluginmatch-
required_toolsrequiredToolsmatch-
step_countstepCountmatch-
tokens_betweentokensBetweenmatch-
tokens_undertokensUndermatch-
tool_arg_satisfiestoolArgSatisfiesmatch-
tool_args_match_schematoolArgsMatchSchemamatch-
tool_call_counttoolCallCountmatch-
tool_called_beforetoolCalledBeforematch-
tool_cardinalitytoolCardinalitymatch-
tool_result_matches_schematoolResultMatchesSchemamatch-
tools_called_exactlytoolsCalledExactlymatch-
tools_called_in_ordertoolsCalledInOrdermatch-
tracetracematch-
trajectorytrajectorymatch-
trajectory_partial_ordertrajectoryPartialOrdermatch-
worldworldmatch-
PythonTypeScriptStatusNote
clientclientmatch-
enforceenforcematch-
evaluateevaluatematch-
evaluate_async-exception (language_idiom)evaluate is already async in TypeScript.
inspectinspectmatch-
inspect_async-exception (language_idiom)inspect is already async in TypeScript.
startstartmatch-
stopstopmatch-
PythonTypeScriptStatusNote
--ci--cimatch-
--json--jsonmatch-
--mcp--mcpmatch-
--skills--skillsmatch-
PythonTypeScriptStatusNote
AttestPluginAttestPluginmatch-
PluginRegistryPluginRegistrymatch-
PluginResultPluginResultmatch-
execute_plugin_assertionexecutePluginAssertionmatch-
load_entrypoint_pluginsloadPackagePluginsalias-
register_pluginregisterPluginmatch-
PythonTypeScriptStatusNote
ADVERSARIAL_USERADVERSARIAL_USERmatch-
CONFUSED_USERCONFUSED_USERmatch-
COOPERATIVE_USERCOOPERATIVE_USERmatch-
FRIENDLY_USERFRIENDLY_USERmatch-
MockToolRegistryMockToolRegistrymatch-
PersonaSimulatePersonaalias-
RepeatResultRepeatResultmatch-
ScenarioConfigScenarioConfigmatch-
ScenarioResultScenarioResultmatch-
configureconfigaliasconfig configures simulation mode in TypeScript.
fault_injectfaultInjectmatch-
is_simulation_modeisSimulationModematch-
mock_toolmockToolmatch-
repeatrepeatmatch-
scenarioscenariomatch-
PythonTypeScriptStatusNote
attestuseAttestaliasThe pytest attest marker and fixture map to useAttest(); Vitest selects tests by file glob.
attest_budgetevaluatealiasPass the testBudget option to evaluate/inspect.
attest_engineuseAttestalias-
attest_min_pass_ratetrialsaliasPass the minPassRate option to trials.
attest_replayuseReplayalias-
attest_tierfilterByTieraliasFilter with filterByTier; tag tests with tier.
attest_worlduseWorldalias-
integration-exception (language_idiom)Vitest splits integration tests with a separate include glob or test:integration script.