Back to API and technical reference
SPARQL 1.1 conformance
How much of SPARQL does rete actually answer correctly? This runs the official
W3C SPARQL 1.1 query-evaluation test suite — the canonical
w3c/rdf-tests fixtures: each test ships a
tiny RDF dataset, a query, and the expected results — against rete and
scores every test. It is a correctness suite (the datasets are a handful of
triples), so it measures coverage, not speed; performance is benchmarked
separately on a real graph (see BENCHMARK).
The harness is scripts/sparql_conformance.py:
for each mf:QueryEvaluationTest it builds a .rete from the data, runs the
query through the rete CLI, and compares to the expected result — SPARQL
Results JSON/XML (SRX/SRJ) for SELECT/ASK as an unordered multiset, and an RDF
graph isomorphism for CONSTRUCT/DESCRIBE.
python scripts/sparql_conformance.py \
--rete target/release/rete --suite <rdf-tests>/sparql/sparql11
Scorecard
312 query-evaluation tests, byte-for-byte against the W3C expected results.
Measured 2026-10-09 (after the correlated zero-length path fix) against w3c/rdf-tests 5e5da96
(2026-10-02) — the revision pinned in
tests/w3c-rdf-tests.rev
(see The pinned suite).
"n/s" = errored / not supported.
| Category | pass | n/s | notes |
|---|---|---|---|
| negation | 12 / 12 | 0 | ✅ full |
| json-res | 4 / 4 | 0 | ✅ full |
| cast | 6 / 6 | 0 | ✅ full — xsd:integer/decimal/float/double/boolean/string |
| bind | 10 / 10 | 0 | ✅ full — in-pattern BIND visible to later FILTER/join |
| grouping | 4 / 4 | 0 | ✅ full |
| bindings (VALUES) | 10 / 12 | 2 | |
| aggregates | 40 / 42 | 2 | GROUP_CONCAT/SUM/AVG/SAMPLE incl. DISTINCT |
| property-path | 30 / 33 | 2 | incl. negated property sets + zero-length on empty data |
| construct | 3 / 5 | 1 | graph-isomorphism check |
| exists | 4 / 6 | 1 | |
| project-expression | 7 / 7 | 0 | ✅ full |
| functions | 73 / 75 | 1 | nearly full — only NOW() + IRI() base resolution |
| entailment | 28 / 70 | 4 | needs build --materialize |
| subquery | 7 / 16 | 7 | nested SELECT joins; GRAPH-scoped + RDF/XML data n/a |
| service | 1 / 7 | 6 | SERVICE is implemented — most of these tests need a live endpoint, which the offline run does not have |
| csv-tsv-res | 0 / 3 | 3 | CSV/TSV result format |
| TOTAL | 239 / 312 (76.6%) | 29 |
This total is measured in CI, not written by hand. The conformance job runs the harness against the W3C suite on every Rust change and fails if the passing count drops below
tests/conformance-baseline.json. It was not always so: the figure here read 232 / 75.1% while the engine actually scored 236 / 76.4%, and nothing would have caught it moving the other way. It then drifted again, to a stale 236 / 309, because only the baseline file is checked, not this page. The suite itself also moved: CI used to clonew3c/rdf-testsat HEAD on every run, so the denominator followed upstream (309 tests in June, 312 by October) and the job could change its verdict with no rete commit. It is pinned now.
The pinned suite
The W3C suite revision lives in one place:
tests/w3c-rdf-tests.rev,
a single full commit SHA of w3c/rdf-tests
— currently 5e5da96bb06c551ae3ac57d30fec8116f7161b9b (2026-10-02, 312
query-evaluation tests). CI checks out exactly that commit (sparse, only
sparql/sparql11, cached by SHA) and fails if what it got is anything else;
tests/conformance-baseline.json names the file in suite_revision_file, and
its counts are only meaningful against that commit. The harness prints the
revision it ran against and warns when it is not the pinned one.
Bumping the suite is a deliberate one-line change — replace the SHA in
tests/w3c-rdf-tests.rev. Do it in its own PR, and in the same PR:
- re-run the harness against the new commit and regenerate the baseline in
place (
--json tests/conformance-baseline.jsonkeeps_commentandsuite_revision_file, and updates only the counts); - update the scorecard above — the date, the SHA and the per-category rows;
- explain any change in
passthat is the suite's doing (new or changed tests) rather than rete's, since the two now move separately.
Coverage notes
Strong areas. Negation (MINUS / NOT EXISTS), JSON results, XSD casts
(with strict lexical validation), and projection expressions are 100%. Property
paths are near-full — including negated property sets (!(:p1|…|:pn)) and the
zero-length identity solution on */? even against empty data. BIND inside
a WHERE pattern is visible to later filters and joins; nested SELECT
subqueries are evaluated independently and joined on their projected variables;
the aggregate set is complete (GROUP_CONCAT with DISTINCT/SEPARATOR,
typed computed numerics, 15-significant-digit decimal round-trips). The
built-in function library covers strings (language-tag-preserving
REPLACE/CONCAT/SUBSTR/…, STRDT/STRLANG, ENCODE_FOR_URI,
LANGMATCHES), the MD5…SHA512 hashes, the xsd:dateTime accessors, and
IF/IN/sameTerm — all in pure Rust, so the same coverage holds in the WASM
client. CONSTRUCT answers are verified by graph isomorphism. This matches the
24-operator cross-check against Oxigraph in BENCHMARK — which
runs in CI as of this change. It had not: the differential oracle lives in the
rete-bench crate, and every CI invocation excluded that crate, so the test
described in its own header as "a correctness gate" ran nowhere while build
reports went on printing "differential oracle green".
Out of scope (counted against the total, but not engine bugs):
- SERVICE (7) —
SERVICEfederation is implemented (the block is sent to the remote endpoint and joined — see sparql); these tests need a live endpoint, so they stay out of the offline suite. (Cross-file federation is a different feature — see federation.) - entailment (≈49) — RDFS/OWL entailment regimes; rete answers these only
when entailments are baked in at build time (
rete build --materialize), which this run does not do. - subquery (12 n/s) — plain nested
SELECTis evaluated (see above); the remaining n/s are GRAPH-scoped subqueries and tests whose data ships as RDF/XML, not the subquery feature itself. - csv-tsv-res (3) — the CSV/TSV result serialization isn't implemented.
Remaining gaps in functions (2): NOW() (no wall clock on the
wasm32-unknown-unknown target the engine must also compile to) and IRI()
relative-base resolution. The non-deterministic builtins RAND,
UUID/STRUUID, BNODE work in the browser too (via getrandom's js
backend), and an error in an IF condition propagates rather than silently
taking the else-branch.
The same answers, lazily and remotely
The conformance run opens each file locally (in memory). rete's three read
modes — local, remote-lazy (HTTP range reads via sparql_url /
sparql-url), and remote-cached (download once, query in memory) — share the
identical evaluator; only the byte source differs. So they return identical
results — demonstrated on a real ~12 M-triple graph: the same query yields the same rows whether served locally,
lazily from a CDN, or federated across shards. The tiny conformance fixtures make
per-mode timing meaningless — that comparison lives in
BENCHMARK, on a graph large enough for it to matter.
Reproduce
REV=$(tr -d '[:space:]' < tests/w3c-rdf-tests.rev) # the pinned suite
git init -q rdf-tests
git -C rdf-tests remote add origin https://github.com/w3c/rdf-tests.git
git -C rdf-tests sparse-checkout set sparql/sparql11
git -C rdf-tests fetch -q --depth 1 --filter=blob:none origin "$REV"
git -C rdf-tests checkout -q FETCH_HEAD
cargo build --release -p rete-cli
python scripts/sparql_conformance.py --rete target/release/rete \
--suite rdf-tests/sparql/sparql11 # add --relaxed for the value column