SPARQL 1.1 conformance

How much of SPARQL does rete actually answer correctly? This runs the official W3C SPARQL 1.1 query-evaluation test suite — the canonical w3c/rdf-tests fixtures: each test ships a tiny RDF dataset, a query, and the expected results — against rete and scores every test. It is a correctness suite (the datasets are a handful of triples), so it measures coverage, not speed; performance is benchmarked separately on a real graph (see BENCHMARK).

The harness is scripts/sparql_conformance.py: for each mf:QueryEvaluationTest it builds a .rete from the data, runs the query through the rete CLI, and compares to the expected result — SPARQL Results JSON/XML (SRX/SRJ) for SELECT/ASK as an unordered multiset, and an RDF graph isomorphism for CONSTRUCT/DESCRIBE.

python scripts/sparql_conformance.py \
  --rete target/release/rete --suite <rdf-tests>/sparql/sparql11

Scorecard

309 query-evaluation tests, byte-for-byte against the W3C expected results. "n/s" = errored / not supported.

This total is measured in CI, not written by hand. The conformance job runs the harness against the W3C suite on every Rust change and fails if the passing count drops below tests/conformance-baseline.json. It was not always so: the figure here read 232 / 75.1% while the engine actually scored 236 / 76.4%, and nothing would have caught it moving the other way.

Coverage notes

Strong areas. Negation (MINUS / NOT EXISTS), JSON results, XSD casts (with strict lexical validation), and projection expressions are 100%. Property paths are near-full — including negated property sets (!(:p1|…|:pn)) and the zero-length identity solution on */? even against empty data. BIND inside a WHERE pattern is visible to later filters and joins; nested SELECT subqueries are evaluated independently and joined on their projected variables; the aggregate set is complete (GROUP_CONCAT with DISTINCT/SEPARATOR, typed computed numerics, 15-significant-digit decimal round-trips). The built-in function library covers strings (language-tag-preserving REPLACE/CONCAT/SUBSTR/…, STRDT/STRLANG, ENCODE_FOR_URI, LANGMATCHES), the MD5…SHA512 hashes, the xsd:dateTime accessors, and IF/IN/sameTerm — all in pure Rust, so the same coverage holds in the WASM client. CONSTRUCT answers are verified by graph isomorphism. This matches the 24-operator cross-check against Oxigraph in BENCHMARK — which runs in CI as of this change. It had not: the differential oracle lives in the rete-bench crate, and every CI invocation excluded that crate, so the test described in its own header as "a correctness gate" ran nowhere while build reports went on printing "differential oracle green".

Out of scope (counted against the total, but not engine bugs):

• SERVICE (7) — SERVICE federation is implemented (the block is sent to the remote endpoint and joined — see sparql); these tests need a live endpoint, so they stay out of the offline suite. (Cross-file federation is a different feature — see federation.)
• entailment (≈49) — RDFS/OWL entailment regimes; rete answers these only when entailments are baked in at build time (rete build --materialize), which this run does not do.
• subquery (12 n/s) — plain nested SELECT is evaluated (see above); the remaining n/s are GRAPH-scoped subqueries and tests whose data ships as RDF/XML, not the subquery feature itself.
• csv-tsv-res (3) — the CSV/TSV result serialization isn't implemented.

Remaining gaps in functions (2): NOW() (no wall clock on the wasm32-unknown-unknown target the engine must also compile to) and IRI() relative-base resolution. The non-deterministic builtins RAND, UUID/STRUUID, BNODE work in the browser too (via getrandom's js backend), and an error in an IF condition propagates rather than silently taking the else-branch.

The same answers, lazily and remotely

The conformance run opens each file locally (in memory). rete's three read modes — local, remote-lazy (HTTP range reads via sparql_url / sparql-url), and remote-cached (download once, query in memory) — share the identical evaluator; only the byte source differs. So they return identical results — demonstrated on a real ~12 M-triple graph: the same query yields the same rows whether served locally, lazily from a CDN, or federated across shards. The tiny conformance fixtures make per-mode timing meaningless — that comparison lives in BENCHMARK, on a graph large enough for it to matter.

Reproduce

git clone --depth 1 --filter=blob:none --sparse https://github.com/w3c/rdf-tests
cd rdf-tests && git sparse-checkout set sparql/sparql11 && cd ..
cargo build --release -p rete-cli
python scripts/sparql_conformance.py --rete target/release/rete \
  --suite rdf-tests/sparql/sparql11            # add --relaxed for the value column