Vector conformance adapters

Typra emits two tiers of generated tests. Both start from the same raw ingredients — a declared surface plus example data — but historically only one was enforced:

Tier	Declared surface	Example data	Generated test	Enforced?
Structural (types)	model	@sample	Serialization round-trip	Yes — mandatory, must pass
Behavioral (operations)	interface / op	@vector	(none — data only)	No

Structural parity is automatic because the guarantee is a generated, mandatory, must-pass test — not a convention anyone has to remember. Behavioral vectors used to stop at data: Typra emitted the interface and the vector payloads, but the generated “test” only compared the vector data to itself, so a runtime could be fully green while implementing none of the operations. Absent tests cannot fail, so behavioral parity drifted silently.

This page is the contract that closes the operation loop the same way the type loop is closed. Per target language, Typra emits a mandatory, runnable conformance suite driven entirely by the single vectors.json. Each vector resolves a runtime-supplied adapter through a required registration hook, invokes it with the vector input, canonically compares the result against expected, and reports per-vector pass/fail. A missing adapter is a build or hard failure — never a silent skip — unless the operation is on an explicit, tracked waiver list.

Author adapters against this contract; it is the only runtime-authored surface. No runtime hand-writes vector iteration, input resolution, or comparison logic.

Where adapters live

The generated conformance suite is emitted into each target’s test-dir and is regenerated (and pruned) on every emit, so runtime code must not live inside output-dir or test-dir. Instead, the runtime authors one adapter module outside the generated tree and points the target at it:

emit-targets:
  - type: TypeScript
    output-dir: "generated/typescript"
    test-dir: "generated/typescript/tests"
    import-path: "../index"
    # Module the generated conformance suite imports its adapters from.
    vector-adapter-path: "../../../runtime/vector-adapters"
  - type: Python
    output-dir: "generated/python"
    test-dir: "generated/python/tests"
    import-path: "todo_contracts"
    vector-adapter-path: "todo_runtime.vector_adapters"

If vector-adapter-path is omitted, the generated suite falls back to a conventional sibling module (vector-adapters / vector_adapters / vectoradapters / vector_adapters.rs / <root-namespace>.Conformance). Either way, if the module cannot be imported the suite fails to compile or collect — a missing adapter surface is a hard failure, not a skip.

The registration contract

Each adapter is conceptually adapter(inputJson, context) -> resultJson: it maps the vector’s JSON input to the real implementation call (matching the operation’s declared signature) and returns a value the harness canonicalizes. Adapters are keyed by the operation id Contract.operation (for example Renderer.render). The runtime registers exactly one adapter per operation it implements.

TypeScript

// runtime/vector-adapters.ts  (hand-authored, outside the generated tree)
import { Renderer } from "../src/renderer";

// Optional: deterministic stand-ins the harness exposes as `context.doubles`.
export const vectorDoubles = { clock: () => 0 };

export const vectorAdapters = {
  // Real protocol interface: thin JSON -> implementation -> JSON mapping.
  "Renderer.render": {
    invoke: (input, ctx) => {
      const renderer = new Renderer({ clock: ctx.doubles.clock });
      return renderer.render(input.request);
    },
  },

  // Conformance-only interface: still required, still a thin adapter.
  "Processor.process": {
    invoke: (input) => Processor.process(input.request, input.response),
  },
};

// Optional: operations Typra treats as tracked-skipped for THIS runtime.
// A map of `Contract.operation` -> reason (embed an issue link in the reason).
export const vectorWaivers = {
  "Ignorer.ignore": "Not yet implemented (org/repo#123)",
};

The generated vector-conformance.test.ts imports the adapter module by path, so in a typed downstream build a missing module is a module-not-found compile error; at runtime a missing module is an import failure. Either way it is a hard failure, never a silent skip.

Python

# todo_runtime/vector_adapters.py  (hand-authored, outside the generated tree)
from todo_runtime.renderer import Renderer

VECTOR_ADAPTERS = {
    "Renderer.render": {
        "invoke": lambda input, ctx: Renderer(clock=ctx["doubles"]["clock"]).render(input["request"]),
    },
    "Processor.process": {
        "invoke": lambda input, ctx: process(input["request"], input["response"]),
    },
}

# Optional: operations Typra treats as tracked-skipped for THIS runtime.
# A dict of `Contract.operation` -> reason (embed an issue link in the reason).
VECTOR_WAIVERS = {
    "Ignorer.ignore": "Not yet implemented (org/repo#123)",
}

The generated test_vector_conformance.py imports VECTOR_ADAPTERS, so a missing module is a collection-time ImportError (hard failure).

Go

// vectoradapters/adapters.go  (hand-authored, outside the generated tree)
package vectoradapters

// VectorError carriers attach the expected payload via TypraVector().
func Adapters() map[string]VectorAdapter {
  return map[string]VectorAdapter{
    "Renderer.render": {Invoke: func(input any, ctx VectorContext) (any, error) {
      return NewRenderer(ctx.Doubles).Render(input)
    }},
  }
}

// Optional: `Contract.operation` -> reason (embed an issue link).
func Waivers() map[string]string {
  return map[string]string{"Ignorer.ignore": "Not yet implemented (org/repo#123)"}
}

The generated vector_conformance_test.go imports the vectoradapters package, so a missing package is a compile error (go test never runs), never a skip.

Rust

// tests/vector_adapters.rs  (hand-authored, outside the generated tree)
#![allow(dead_code)]
use std::collections::HashMap;
use serde_json::Value;

pub fn adapters() -> HashMap<&'static str, Adapter> {
    let mut m = HashMap::new();
    // Sync adapter: a bare `fn` returning a plain value — no boxing, no bridge.
    m.insert("Renderer.render", Adapter::sync(render_adapter));
    // Async adapter: `Adapter::asynchronous` boxes the future for you, so the
    // body is a plain `async move`. Own your inputs inside it (the future is
    // `'static`): clone from the borrowed `&Value`/`&Context` as needed.
    m.insert("Processor.process", Adapter::asynchronous(|input, ctx| {
        let input = input.clone();
        async move { Ok(process(&input.request).await) }
    }));
    m
}

// Optional: `Contract.operation` -> reason (embed an issue link).
pub fn waivers() -> HashMap<&'static str, &'static str> {
    HashMap::new()
}

An adapter is registered as either Adapter::sync(fn) (a synchronous fn that returns Result<Value, VectorError>) or Adapter::asynchronous(closure) (a closure returning any future resolving to the same). The #[tokio::test] harness awaits it exactly once on a current-thread runtime; a sync adapter resolves without ever touching the runtime. For an error vector attach the expected payload as VectorError.payload. The generated integration-test crate declares the adapter module via #[path = ...], so a missing module is a compile error under cargo test, never a skip. Add a dev-dependency on tokio (with the macros and rt features) so the async test attribute resolves. The future is not required to be Send.

C#

// Conformance/VectorAdapters.cs  (hand-authored, outside the generated tree)
namespace Todo.Contracts.Conformance;

using System.Text.Json.Nodes;

public static class VectorAdapters
{
    public static Dictionary<string, VectorAdapter> Adapters() => new()
    {
        ["Renderer.render"] = new VectorAdapter
        {
            Invoke = (input, ctx) => Renderer.Render(input, ctx.Doubles),
        },
    };

    // Optional: `Contract.operation` -> reason (embed an issue link).
    public static Dictionary<string, string> Waivers() => new();

    public static JsonNode? Doubles() => null;
}

Invoke returns a JsonNode?; for an error vector throw a VectorException whose Payload (a JsonNode?) carries the expected error. The generated VectorConformanceTests.cs has using <vector-adapter-path>;, so a missing adapter namespace is a compile error under dotnet test, never a skip.

Java

// typra/proof/adapters/VectorAdapters.java  (hand-authored, outside the generated tree)
package typra.proof.adapters;

import com.fasterxml.jackson.databind.JsonNode;
import java.util.HashMap;
import java.util.Map;
import typra.proof.VectorRunner.VectorAdapter;
import typra.proof.VectorRunner.VectorContext;
import typra.proof.VectorRunner.VectorException;

public final class VectorAdapters {
    private VectorAdapters() { }

    public static Map<String, VectorAdapter> adapters() {
        Map<String, VectorAdapter> m = new HashMap<>();
        m.put("Renderer.render", new VectorAdapter(Renderer::render));
        return m;
    }

    // Optional: `Contract.operation` -> reason (embed an issue link).
    public static Map<String, String> waivers() { return new HashMap<>(); }

    public static JsonNode doubles() { return null; }
}

The support types (VectorAdapter, VectorContext, VectorException, and the Invoke/Normalizer functional interfaces) are nested inside the generated VectorRunner (the interpreter module beside the thin VectorConformanceTests harness), so import them as <package-name>.VectorRunner.VectorAdapter etc. invoke returns a JsonNode; for an error vector throw a VectorException whose payload (a JsonNode) carries the expected error. The Java harness is only emitted for the jackson serialization mode, which supplies the JSON runtime it parses and canonicalizes with; a missing adapter class is a compile error, never a skip.

Swift

// Tests/.../Adapters.swift  (hand-authored, outside the generated files)
import Foundation

enum VectorAdapters {
    static func adapters() -> [String: VectorAdapter] {
        var m: [String: VectorAdapter] = [:]
        m["Renderer.render"] = VectorAdapter { input, ctx in
            try Renderer.render(input, ctx.doubles)
        }
        return m
    }

    // Optional: `Contract.operation` -> reason (embed an issue link).
    static func waivers() -> [String: String] { return [:] }

    static func doubles() -> Any? { return nil }
}

The support types (VectorAdapter, VectorContext, VectorError) are declared in the generated VectorConformanceTests.swift, so the adapter enum is authored as a sibling non-generated file in the same XCTest target and needs no import. An invoke closure returns Any?; for an error vector throw a VectorError whose payload (an Any?) carries the expected error. The harness parses and compares with Foundation’s JSONSerialization, so it is emitted regardless of serialization mode; a missing adapter enum is a compile error under swift test, never a skip.

Adapter shape

Field	Required	Purpose
invoke(input, context)	Yes	Maps opaque JSON input to the real call; returns a JSON-able result (sync or async).
normalize(result, context)	No	Per-operation normalizer for inherently nondeterministic output (see Normalization).

input and expected stay opaque and are compared canonically, never by typed equality. The adapter maps JSON ↔ real implementation; it must not force the runtime to expose internals beyond what the call needs.

For an error vector (one that declares expectedError instead of expected), the adapter must raise/throw. The harness compares the failure against expectedError canonically: attach the expected payload as typraVector (TypeScript) / typra_vector (Python) on the thrown error, TypraVector() (Go), VectorError.payload (Rust), VectorException.Payload (C#), VectorException.payload (Java), or VectorError.payload (Swift) — or return it from a normalize hook. An adapter that returns a value where an error was expected is a failure.

Async adapters

invoke may return either a plain value or a language-native awaitable, and the harness awaits (unwraps) the result before normalizing and comparing. This lets an adapter drive the runtime’s real async code path directly on the test framework’s own event loop, instead of forcing a synchronous blocking bridge (asyncio.run / block_on / .GetAwaiter().GetResult() / a semaphore) that tests the runtime in a context users never hit.

The generated suites use each framework’s native async test form and apply an “await-if-awaitable” unwrap:

Target	Test form	Unwrap
TypeScript / JS	it(name, async () => …)	await adapter.invoke(...) (a no-op on a non-Promise)
Python	async def test_… under pytest-asyncio	if inspect.isawaitable(r): r = await r
C#	async Task (xUnit)	await if the result is a Task/ValueTask
Rust	#[tokio::test] async fn	Adapter::sync(fn) or Adapter::asynchronous(closure); the harness awaits once
Swift	func … () async throws (XCTest)	try await adapter.invoke(...)
Go	plain function call	none — Go has no awaitable type
Java	JUnit method	join if the result is a Future/CompletableFuture

A synchronous adapter needs no changes: awaiting or unwrapping a non-awaitable degrades to the plain value in every target, so existing discovery adapters stay green untouched. Error-path parity is preserved — an awaitable that rejects or completes exceptionally is compared against expectedError exactly as a synchronous throw is.

The adapter contract is: perform exactly one awaited invocation per vector and spawn no adapter-managed concurrency of your own, so conformance stays deterministic.

@sync is enforced, not advisory

Operations are async-capable by default. The default is permissive: an adapter may return a plain value or an awaitable, and both pass. Marking an operation @sync turns synchronous resolution into a hard behavioral guarantee — the harness rejects a @sync adapter that hands back an awaitable, because a @sync operation must resolve without a runtime hop. This keeps the classification honest across every runtime instead of letting one target quietly wire an async body behind a sync signature.

The check runs before the expectedError comparison and raises a distinct failure (never swallowed as an observed error), keyed off each language’s native awaitable shape:

Target	@sync violation is…
TypeScript / JS	a returned Promise/thenable
Python	an inspect.isawaitable result
C#	a returned Task/ValueTask
Rust	registering the adapter Adapter::asynchronous(...) (the enum tag is the classification)
Swift	registering the adapter with the asynchronous: initializer
Java	a returned Future/CompletableFuture
Go	not enforced — Go has no awaitable type, so a sync signature is the only shape a value can take

The reverse is deliberately not enforced: an async-default operation whose adapter happens to resolve synchronously still passes, so no discovery adapter needs a rewrite. To make a @sync operation async-capable, drop @sync; to keep @sync, make the adapter resolve synchronously.

Collaborator injection seams

Some operations do not run in isolation — they select a strategy per backend/variant, or depend on something nondeterministic (a clock, an id generator, a remote service). The context passed to every adapter carries the seams that keep results reproducible:

context member	Purpose
operation / contract	The operation being exercised and its contract.
vector	The full vector entry (name, stage, and any metadata).
provider / targetApi	The vector-indicated variant, so one adapter can dispatch to the right backend.
doubles	Deterministic stand-ins from the module’s optional vectorDoubles / VECTOR_DOUBLES export (fixed clock, fixed id generator, mock responders).
baseDir	Directory external input references resolve against (the generated test directory).
resolveInput(value)	Resolves external input references (see below).

The runtime supplies deterministic doubles either by closing over them directly in the adapter or by exporting vectorDoubles / VECTOR_DOUBLES, which the harness forwards as context.doubles. The harness passes the vector-indicated provider so a single adapter can behave differently per variant without the harness knowing anything about the runtime’s internals.

Input resolution

Vectors may reference external inputs instead of embedding them. A reference is a JSON object with a single resolver key; the harness resolves it (via context.resolveInput, applied automatically before invoke) relative to context.baseDir — the generated test directory by default, so external inputs are co-located deterministically with the suite:

Reference	Resolves to
{ "$file": "fixtures/prompt.txt" }	UTF-8 contents of the referenced file.
{ "$json": "fixtures/request.json" }	Parsed JSON contents of the referenced file.
{ "$env": "OPENAI_BASE_URL" }	The named environment variable’s value (empty string when unset).

Resolution is centralized in the generated harness; runtimes never re-implement it.

Normalization

Actual and expected results are compared in one canonical form, defined once in the generated harness so no runtime reinvents it:

• object keys sorted;
• numbers compared by value (no insignificant-digit or -0 drift);
• no insignificant whitespace;
• array order is significant unless a per-operation normalizer sorts it.

When an operation’s output is inherently nondeterministic (timestamps, generated ids, map ordering), register a per-operation normalize(result, context) hook on the adapter. Prefer deterministic doubles first; use normalize only for output the runtime genuinely cannot pin.

Waivers and the skip ramp

Every @vector-bound operation must be implemented or carry a tracked waiver. There is no blanket “skip all.”

• An operation with a registered adapter runs and must pass.
• An operation with no adapter and no waiver makes the generated suite hard-fail (or fail to compile/collect).
• An operation whose key appears in the runtime’s vectorWaivers / VECTOR_WAIVERS map emits as a visible tracked-skip — a SKIP line in the TypeScript suite and a real pytest.skip (an s) in the Python suite. Each waiver maps a concrete Contract.operation to a reason string; wildcards are rejected.

Waivers keep CI green during migration while the map visibly shrinks toward zero. The same invariant is available as a static gate: evaluateVectorAdapterCoverage (in src/ir/vector-coverage.ts) checks a runtime’s declared adapter/waiver keys against vectors.json and reports any operation that has a vector but neither an adapter nor an enumerated waiver, so a missing adapter is caught ahead of the run. Adding a vector without a corresponding adapter (and without a waiver) cannot merge.

typra-verify wires this gate in: declare the runtime’s implemented operations in the verifier config and the gate runs against the current vectors.json snapshot as part of metadata verification.

typra-verify.config.json

{
  "vectorAdapters": ["Renderer.render", "Processor.process"],
  "vectorWaivers": ["Ignorer.ignore"]
}

An operation carrying a vector but absent from both lists raises a blocking vector-adapter-coverage failure (ok: false); a wildcard waiver is rejected outright (both when the config is loaded and by the gate itself). The verify summary reports vector coverage: <covered>/<operations> covered, <waived> waived, <missing> missing.

The config declares the runtime’s intent — it is a consistency check, not proof that the named adapters exist. The generated conformance suite remains the real runtime enforcement (a declared-but-unimplemented adapter still fails there), so keep that suite in CI alongside typra-verify.

Requirement guards

A vector may declare an ordered list of abstract capability tokens it needs before it can meaningfully run:

const LiveVectors = #[
  #{
    name: "structure",
    input: #{ request: #{ prompt: "hi" } },
    expected: #{ role: "assistant" },
    requires: #["provider:openai", "entra:foundry-project"]
  }
];

The guard is a general primitive: conditionally skip a conformance vector when a capability it depends on is absent, uniformly across all seven runtimes, instead of seven hand-authored invoke-side skips. The motivating case is live-provider vectors that call a real API and assert structure — when the credential is absent the vector must self-skip rather than send an empty credential and fail — but the same mechanism gates on any capability a runtime can probe (a reachable service, a present license, a feature flag). The emitter knows nothing about what a token means; it only wires the skip.

Token grammar

Tokens follow a namespace:name convention (both segments lowercase, hyphen-separated: [a-z0-9][a-z0-9-]*) — for example provider:openai, entra:foundry-project, or a domain-neutral service:database / feature:experimental. The emitter treats tokens as opaque strings; it never parses the namespace. Env-var presence is just one predicate flavor a runtime may implement — entra:foundry-project is a probe (can we mint an Entra token for a project URL?), not an env-key check. The grammar is an authoring convention, enforced only by the decorator’s “non-empty string” shape check.

The VECTOR_CAPABILITIES seam

Alongside VECTOR_ADAPTERS and VECTOR_WAIVERS, a runtime supplies a capability table in the same module named by vector-adapter-path: a map from a token string to a predicate. The predicate receives the same context object the adapter receives and returns truthy when the capability is available.

Runtime	Accessor	Predicate
Python	VECTOR_CAPABILITIES	token -> callable(context) -> bool
TypeScript	vectorCapabilities	(context) => boolean
Go	VectorCapabilities	map[string]func(Context) bool
C#	VectorAdapters.Capabilities()	Func<VectorContext, bool>
Java	VectorAdapters.capabilities()	Capability (boolean test(VectorContext))
Swift	VectorAdapters.capabilities()	(VectorContext) -> Bool
Rust	vector_adapters::capabilities()	fn(&Context) -> bool

The seam is emitted per suite, on demand: the harness loads VECTOR_CAPABILITIES and emits the guard only when at least one vector in that suite declares requires. A suite with no requires tokens generates no capability load and no guard, so existing runtimes need not add an (empty) capability table until they actually author a requires vector. This keeps regeneration a clean no-op for suites that don’t use the feature.

Ordering and the canonical skip reason

The guard runs after adapter resolution (so a normal op with a missing adapter still hard-fails) and before the per-vector waiver check and the adapter invoke. It makes two deterministic passes over requires, in author order (never sorted):

1. Registration pass — every token must be present in the capability table. An unregistered token is a hard failure (see below). This runs first so a typo always fails even when a later token is also absent.
2. Availability pass — predicates evaluate in order; on the first falsy result the harness emits its language’s skip with the canonical, byte-identical reason requirement unavailable: <token> (a pytest.skip/t.Skip where the framework supports it, a SKIP … (requirement unavailable: <token>) recorded line otherwise).

Unknown tokens fail hard

A token with no registered predicate is a hard failure, never a silent skip:

No capability predicate registered for requirement token “<token>”. Register it in the module referenced by vector-adapter-path. @vector conformance never skips silently.

Otherwise a typo would quietly disable a live test on every runtime.

Three distinct axes

Keep the three outcomes separate — the reason strings distinguish them in CI logs:

• waiver — a permanent, tracked gap (VECTOR_WAIVERS), reason waived: ….
• capability-absent — a transient environment condition (this guard), reason requirement unavailable: ….
• failure — adapter present, capability present, wrong structure.

Rust caveat

Rust has no runtime-conditional skip (#[ignore] is compile-time). The Rust harness therefore treats an unavailable capability as a best-effort skip: println!("SKIP {} (requirement unavailable: {})", …); return; — the test passes and records intent on stdout, exactly as the Rust harness already handles waivers. An unknown token still panic!s. “Uniform skip” is thus best-effort in Rust.

Doctrine

Any interface documented as “not expected to be implemented by any runtime” is inverted by this contract: every target must implement each @vector-bound operation or declare a tracked waiver. Conformance-only interfaces are still required — they simply tend to have the thinnest adapters. See Interfaces, operations, and transport and Typra contracts for how vectors lower into the callable IR that this contract consumes.