WASM & JavaScript API

crates/rete-wasm compiles the same engine (dictionary, permutation indexes, SPARQL, zstd decode) to WebAssembly, so a web page queries a .rete file client-side with no server. web/index.html is a working serverless explorer.

Build

docker compose build wasm
docker compose run --rm wasm

This runs scripts/build_wasm.sh: it builds web/pkg, web/pkg-nomodules, and web/pkg-nomodules-async from one checkout, regenerates the playground, and writes wasm-build.json with the source revision, pinned tool versions, sizes, and SHA-256 digests. The gitignored embedded .rete datasets listed in scripts/build_playground.py must already be staged under web/.

When the checkout is a Windows Git worktree mounted into Linux, pass the revision because the worktree's host .git indirection is not visible:

docker compose run --rm -e RETE_SOURCE_REVISION=$(git rev-parse HEAD) wasm

The image carries Binaryen v108 for the Asyncify transform. That version corrupts modern wasm-bindgen extern-reference tables, so regular web and no-modules builds deliberately use wasm-pack --no-opt; Rust's release profile still optimizes them. The Asyncify build disables reference types before its explicit, pinned Binaryen transform.

zstd's C encoder isn't used on wasm; decoding uses the pure-Rust ruzstd, so the browser reads compressed files fine. rete-wasm depends on rete-core with --no-default-features.

Stable JS API

The 1.0 stable surface is Graph, RemoteGraph, build, query, query_sparql, header_ranges, summary_overview, and the validation and reasoning functions documented below. reach_parallel and the threads feature are experimental; the Asyncify artifact is an alternative transport, not a different graph/query API.

Functions take file bytes as Uint8Array and return JSON strings. Every Rete-owned JSON object envelope includes "schemaVersion": 1; compatibility functions that return a bare array (graph_names, query_triples, query_sparql, searches, communities, and reachability) remain arrays. Binding failures throw JavaScript Error objects rather than strings.

Local files, read lazily

A File the user picked can be range-read exactly like a URL. Register the blob under an address whose scheme is rete-local:, then pass that address to any *_url function or to new RemoteGraph(…):

// inside a Web Worker (register_local_file's reads use FileReaderSync)
wasm_bindgen.register_local_file("rete-local:1/graph.rete", file);
const g = new wasm_bindgen.RemoteGraph("rete-local:1/graph.rete");
JSON.parse(g.query("SELECT ?p ?o WHERE { <…/s42> ?p ?o }", "table"));
g.stats();   // {fileLength, bytes, requests} — how little was actually read

It is the same reader as the HTTP one with a different bottom transport: Blob.slice() + a synchronous FileReaderSync instead of a ranged GET. The header-window cache, range batching, block cache and polyglot detection all apply unchanged, and stats() counts blob reads the way it counts requests.

Three things follow from that, and all three matter:

• Worker-only, like every other lazy read here — FileReaderSync does not exist on the main thread.
• Both engine builds work. A local read never suspends, so the asyncify variant never routes it through env.rete_fetch_ranges.
• A registration belongs to one wasm instance. Rebuild the worker (an engine switch, a trap, a memory reclaim) and the map starts empty — re-register the same address before opening again.

register_local_file rejects an address that does not begin with rete-local:, so an ordinary URL can never be silently answered from a blob.

Below ~128 MB the playground still reads a local file whole: that path is faster when the query touches everything, and the tabs needing a resident graph (Explore, Map, Build) depend on it. Above it, the whole-file read is what kills the tab, so lazy is the default. Override with localStorage.localLazyAboveMB (0 forces lazy for every local file) — the browser's counterpart to the CLI's RETE_LOCAL_LAZY_ABOVE_MB.

query runs SELECT / ASK / CONSTRUCT / DESCRIBE via eval_query and returns a single JSON envelope with a kind field:

• SELECT → { "schemaVersion":1, "kind":"select", "vars":[…], "rows":[ {var:value,…} ] }
• ASK → { "schemaVersion":1, "kind":"ask", "boolean": true|false }
• CONSTRUCT/DESCRIBE → { "schemaVersion":1, "kind":"construct", "format":"ttl"|"jsonld", "text":"…" } when format is "ttl"/"jsonld", else { "schemaVersion":1, "kind":"construct", "triples":[[s,p,o],…] }.

Graph.query_reasoned(query, format) is query with OWL 2 QL entailment on (see Reasoning); Graph.query_opts(query, format, reason, union) — and the same two methods on RemoteGraph — makes both opt-in evaluation toggles explicit: reason is the entailment switch, union the non-standard union default graph mode, where a pattern outside GRAPH matches the merge of the default graph and every named graph. Both default to off, and plain query never applies either. One cost to know: on a RemoteGraph, a union query over a many-graph file may fault the index tiles of every named graph the merge touches (the empty-default + single-named-graph shape stays a zero-copy borrow), so the merge is strictly per-query opt-in.

communities recomputes the Louvain community decomposition (optionally at a given dendrogram round) and returns per-community member and triple counts — the data behind the playground's "split by community" view.

reach computes multi-source transitive reachability over one predicate. seeds is a JSON array string of seed IRI tokens (e.g. '["<http://ex/app>"]'); a single bare IRI is also accepted. With reverse=true it traverses edges backward ("who reaches the seed?" — impact analysis). It returns one entry per seed in input order: { seed, count, reached }, or { seed, error } for a seed not in the graph (so one unknown seed never fails the whole call). It runs serially — the browser engine is single-threaded; the native CLI's rete reach --parallel fans one task per seed for a real speedup.

build is the ingest path in reverse direction from everything above: it takes RDF text (format: "nt" N-Triples, "nq" N-Quads — named graphs become a dataset — or "ttl" Turtle) and assembles a complete .rete file image in the browser: dictionary, permutation indexes, and the community pyramid. The bytes it returns are immediately queryable by every other function, and downloadable as a file. One caveat: the wasm engine ships only the pure-Rust zstd decoder, so in-browser builds write uncompressed sections (codec NONE) — every reader accepts them, but rete build produces a smaller file from the same input. This powers the playground's Build tab.

sparql_url runs full SPARQL against a remote .rete URL without downloading it: it reads the header, the dictionary chunk directories and index tile directories, then faults in only the dictionary chunks and index tiles the query touches — and full scans coalesce adjacent tiles into batched range reads, so even ?s ?p ?o costs a handful of requests, not one per tile. The result envelope is the same as query, plus a remote object reporting exactly how little of the file was fetched.

The design constraint, honestly: the engine is synchronous, and wasm cannot block on fetch. Instead of an async engine refactor, the byte-range reads use synchronous XHR — which browsers permit only inside Web Workers. So call sparql_url from a worker (see web/sparql-url-worker.js and the "Remote SPARQL" section of the demo page); on the main thread the browser throws. The host must answer Range requests with 206 Partial Content (a host that ignores Range is rejected loudly, never silently mis-read) and send CORS headers when cross-origin. A range fetch that fails mid-query is an error — never a silently incomplete result.

The length probe uses a one-byte ranged GET (reading the total from Content-Range) rather than HEAD, since some hosts reject HEAD — notably Hugging Face's signed-redirect storage, which answers 405.

Caching remote reads

Range reads are cached, so re-running or refining a query on a remote dataset re-fetches almost nothing. Two layers:

• Within a query — a block cache. The lazy reader wraps the raw HTTP-range backend in a BlockCacheReader: every read is served from a 64 KiB aligned block, fetched once and kept. A query's scattered tile reads that fall in the same block cost a single fetch (and a multi-range host coalesces the block fetches further). This works over any single-range backend — S3, a CDN — not just a multi-range gateway.
• Across queries — a resident session. RemoteGraph opens a URL once and keeps the Rete resident, so the block cache and the faulted index tiles and the decoded dictionary chunks all survive between queries. The playground's worker holds one RemoteGraph per URL, so exploring a remote dataset — refining a filter, paging entity tables, re-running — reuses everything already fetched: a fully cached re-run fetches 0 bytes, and the result line shows "0 new bytes, all served from this session's cache" with the cache's size. (The free sparql_url opens a fresh file each call, so it gets only the within-query block cache — use RemoteGraph for cross-query reuse.)

RemoteGraph.stats() returns the session's cumulative { schemaVersion, fileLength, bytes, requests }; the worker diffs successive calls to report one query's physical traffic versus the running session total.

• Across reloads and sessions — an opt-in persistent range cache. The playground's Settings → Persist fetched ranges across reloads toggle installs a tiny XMLHttpRequest shim in each engine worker that mirrors fetched bytes into IndexedDB in 1 MiB blocks (keyed by the file's origin + path) and warms them back on the next load — so a reload re-fetches nothing already held, and the cache survives browser sessions until cleared. Settings shows a per-file breakdown — each cached .rete with how much of it is held (e.g. 16 MB / 1.04 GB · 1.6%) and a fill bar — plus per-file and global Clear. It is off by default, so the default read path is byte-identical with the shim absent.
• Whole-file cache mode — one download, then local. Choosing Cache for a catalog dataset downloads that complete .rete once and stores it in the playground's IndexedDB whole-file store. Queries open those bytes in memory; reloads and later browser sessions reuse the stored file with zero network reads. The entry appears under Settings → Whole-file caches, where it can be removed. If IndexedDB is unavailable or quota is exhausted, the current page still uses its in-memory copy, but a later reload must download it again.

Host CORS, in practice. Range-querying from the browser needs a host that serves the bytes directly to a cross-origin browser request. A plain static server with CORS works; an S3/R2/GCS bucket with CORS works; same-origin always works. What does not work is Hugging Face's buckets/.../resolve endpoint: it returns 405 to a cross-origin browser GET even though it serves the rete CLI fine (the CLI sends no Origin). The bytes themselves are reachable — the resolved signed CDN URL answers 206 to the browser — but the resolve hop refuses browser requests, so the lazy backends can't follow it. A client should probe its data URL on load and surface a clear banner when the host isn't browser-reachable; point at a CORS-enabled direct host to light up the remote backends.

Production catalog contract. Playground datasets are served directly from https://data.graphplaza.com (Cloudflare R2), one folder per dataset, without a redirect or token. Every published .rete must use stable format generation 1 (header byte 0x05) and expose Content-Range, Content-Length, Accept-Ranges, and ETag to CORS callers. web/datasets.lock.json pins each catalog object's size, content hash, and format byte. Run uv run python scripts/check_dataset_catalog.py --all before a release to probe all URLs with a browser-style Range request and verify the lock.

Parallel range reads (opt-in). Sequential synchronous XHR serialises a query's round trips. With cross-origin isolation the explorer can read the faulted ranges in parallel: a pool of fetch workers pulls them (each a synchronous XHR, parallel across the pool) into a SharedArrayBuffer, Atomics-coordinated, and the engine blocks until they land — read_at falls back to sequential when isolation is unavailable, so there is never a regression. Static hosts don't send COOP/COEP, so it is opt-in via ?parallel=1 (a bundled coi-serviceworker.js injects the headers and the page reloads once); the default page stays un-isolated so the cross-origin DuckDB-WASM / SQLite backends keep working.

why_triples exposes the same result-provenance path as rete why. It resolves the optional triple pattern through Rete::query_with_provenance and returns browser-style camelCase fields: resultCount, matchedPattern, indexPermutation, indexSection, dictionaryRange, indexRange, indexSectionRange, and pyramidRange. indexRange is the full permutation container; indexSectionRange is the selected permutation payload inside it. Tile provenance reports the physical tile holding each match — { "available": true, "id": "SPO/3", "range": { … } }.

Minimal example

import init, { info, query_sparql } from "./pkg/rete_wasm.js";
await init();
const bytes = new Uint8Array(await (await fetch("/data.rete")).arrayBuffer());
console.log(JSON.parse(info(bytes)));
const rows = JSON.parse(query_sparql(bytes,
  `PREFIX e: <http://ex/> SELECT ?y WHERE { e:Alice e:knows ?y }`));

Progressive loading (overview without the index)

Three small range requests — header + dictionary + summary — build the coarse graph. On the published davidrumsey.rete that is 17.4 MB of 74.8 MB (23.3%); the 42.6 MB of permutation indexes are never downloaded.

header_ranges + summary_overview implement the "overview first" path in the browser: read the 1 KB header (bytes 0..1024), learn where the dictionary and pyramid summary live, range-fetch only those, and compute the coarse graph — the large triple index is never downloaded.

import init, { header_ranges, summary_overview } from "./pkg/rete_wasm.js";
await init();

const range = async (off, len) => new Uint8Array(await (await fetch(url, {
  headers: { Range: `bytes=${off}-${off + len - 1}` }
})).arrayBuffer());

const total  = +(await fetch(url, { method: "HEAD" })).headers.get("content-length");
const header = await range(0, 1024);
const r      = JSON.parse(header_ranges(header));

const buf = new Uint8Array(total);                 // index region left zero
buf.set(header, 0);
buf.set(await range(r.dictOffset, r.dictLen), r.dictOffset);
buf.set(await range(r.pyramidOffset, r.pyramidLen), r.pyramidOffset);

const overview = JSON.parse(summary_overview(buf)); // index never fetched

This is the same path as rete summary-url natively. It's verified end-to-end in rete-wasm's Node test: with the index region zero-filled, the overview still computes — 3 ranges, and 23.3 % of the file on davidrumsey.rete.

progressive_query uses the same summary-only path for query answering. It is intentionally conservative and returns an error unless the query is exactly one of these shapes:

• SELECT (COUNT(*) AS ?n) WHERE { ?s <predicate> ?o }
• SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }
• SELECT ?p (COUNT(*) AS ?n) WHERE { ?s ?p ?o } GROUP BY ?p
• SELECT DISTINCT ?p WHERE { ?s ?p ?o }
• SELECT (COUNT(DISTINCT ?p) AS ?n) WHERE { ?s ?p ?o }
• ASK { ?s ?p ?o }
• ASK { ?s <predicate> ?o }

Successful responses reuse the normal query envelopes and add progressive, for example:

{
  "schemaVersion": 1,
  "kind": "select",
  "vars": ["n"],
  "rows": [{ "n": "\"42\"^^<http://www.w3.org/2001/XMLSchema#integer>" }],
  "progressive": {
    "stage": "summary",
    "exact": true,
    "readsIndex": false,
    "queryShape": "predicate_count",
    "bytes": 9182,
    "requests": 3,
    "fileBytes": 37210
  }
}

The demo page / playground

docs/playground.html is the static console build. It is generated from web/playground.template.html plus the source fragments in web/playground-src/, then inlines the no-modules WASM glue, WASM bytes, and bundled .rete datasets. It opens directly from file://, defaults to SPARQL, and keeps SHACL, reachability, schema, and provenance modes available without a runtime server or bundler. The WASM initializer receives embedded bytes; the generator removes wasm-bindgen's URL/fetch fallback so app boot cannot silently go to the network.

Beyond the bundled datasets it also opens remote .rete files lazily over HTTP range (a 120 MB / 1 GB graph stays interactive because only the touched tiles cross the wire), caches those reads across queries (above), and federates a query across several sources via the SPARQL console's + Add source button — see Federated queries.

Rich result cells and focused cards

Every Table column and Card field has the same explicit render-type menu. In addition to Auto, Text, Link, Button, Image, IIIF, PDF viewer, Map, 3D, Audio, Video, Spin, and Number, two opt-in types handle values that Auto deliberately does not guess:

• Page preview lazily embeds a desktop-shaped thumbnail as the cell nears the viewport. The iframe uses sandbox="allow-scripts" and referrerpolicy="no-referrer"; it cannot gain same-origin access or navigate the parent. A site's X-Frame-Options or CSP frame-ancestors policy can still block it, so the cell always retains an Open page ↗ link.
• Markdown renders an RDF literal as headings, paragraphs, lists, blockquotes, emphasis, links, and inline/fenced code. Raw HTML remains escaped, and only http:, https:, and mailto: links become anchors. The RDF language badge is preserved.

Every media cell keeps a contextual source link below the preview—Open image, PDF, audio, video, 3D, viewer, manifest, or page—even when its inline preview fails. Clicking an inline PDF page opens a larger paged modal without opening the document a second time; the modal reuses the same PDF.js document and current page. A linearized (Fast Web View) PDF on a server with byte-range support gives the quickest first page. Ordinary PDFs remain compatible: PDF.js may first fetch the file tail or fall back to downloading the complete file. Cross-origin range streaming also requires suitable CORS response headers.

On a fine-pointer desktop, image hover previews grow up to the available viewport but are disabled inside the focused-card dialog. The focused Card view is wider and leaves neighboring cards visible. Navigate it with Prev/Next, Left/Right, native horizontal trackpad scrolling, Shift+wheel, dragging a non-interactive card area, or the visible horizontal scrollbar. Touch devices retain native swipe and vertical scrolling inside long cards.

Find a term

The console's 🔎 Find a term button opens a picker so you don't have to know a graph's IRIs by heart. It opens on the schema card's classes and predicates (instant, from the resident card) and, as you type, also searches entities by label — synchronously for an embedded graph, and over HTTP-range reads (with a spinner) for a remote-lazy one, using the bounded label index. Click any result to drop its <IRI> at the cursor.

Each predicate row also carries a values › drill: click it for a faceted browse of the distinct objects that predicate takes — IRIs resolved to their human labels, literals shown verbatim — then click one to insert it into the query. The values for a predicate are read once and cached, so re-opening is instant; on a remote graph the read is a single bounded range query. The Label selector at the top chooses which predicate is read as the human label (rdfs:label, skos:prefLabel, schema:name, … or auto), and the same choice drives the editor's inline Labels decode chips.

Media & companions

Inline rendering of images / IIIF / 3D / audio / geo cells in playground results — and the asset-preparation pipelines behind them — moved to their own page: Media & SQL companions.