rete · Guides Python — rete-graph

Python API

rete-graph is the Python client for .rete files: native bindings (PyO3) to the same Rust engine behind the CLI and the browser playground. It opens a graph from a local path, an HTTP(S) URL, in-memory bytes, or a custom reader object and queries it with SPARQL — remote files are read lazily over HTTP Range requests, so a selective query over a multi-GB file fetches kilobytes, never the file.

uv pip install rete-graph          # or: pip install rete-graph

Works anywhere real CPython ≥ 3.9 runs: scripts, Jupyter, marimo (desktop/server), Colab, uv/pip/conda environments, on Linux, macOS, and Windows. From 0.2.0 there are also Pyodide wheels for browser Pythons — see JupyterLite & marimo WASM below. A runnable Jupyter notebook tour covers everything below with captured outputs.

Open a graph

import rete_graph as rete

g = rete.open("https://data.graphplaza.com/boe/boe-2026-07-07.rete")     # remote, lazy
g = rete.open("data/example.rete")                            # local file, lazy too
g = rete.open(file_bytes)                                     # bytes image, eager
g = rete.open(url, headers={"Authorization": "Bearer ..."})   # authed hosts

All four paths return the same Graph. Remote and local opens are lazy: the header, dictionary directory, and index tile directories load up front; tile payloads fault in per query and stay cached on the handle, so repeated queries get faster. The host serving a remote file should answer Range requests with 206 Partial Content (any S3/R2/CDN/GitHub URL does — see Hosting your .rete). A host that ignores Range and sends the whole file (200) is served from that body when it is at most 256 MiB, and refused by name above that; any other answer is a loud error, never a silently wrong slice. Remote reads go through the CLI's own HTTP reader: kept-alive connections, up to 16 ranges in flight, nearby ranges merged, transient failures retried, tuned by the same RETE_HTTP_* variables (CLI: Over HTTP). The Pyodide build reads through the browser instead (below) and still requires 206.

g.stats() reports the physical traffic since open — {"fileLength": ..., "bytes": ..., "requests": ...} — the number that makes the lazy story visible: a LIMIT 3 scan of a 6.9 MB remote file fetches ~24% of it in ~13 requests; a cache-hit re-run adds ~0.

Custom readers (fsspec, S3, anything)

Any object with read_at(offset, length) -> bytes and a length (a len() method or __len__) can back a graph — so fsspec reaches authenticated S3/GCS/Azure with no rete-specific code:

import fsspec, rete_graph as rete

class FsspecReader:
    def __init__(self, url, **kw):
        self.f = fsspec.open(url, "rb", **kw).open()
        self.size = self.f.size
    def len(self):
        return self.size
    def read_at(self, offset, length):
        self.f.seek(offset)
        return self.f.read(length)

g = rete.open(reader=FsspecReader("s3://my-bucket/data.rete"))

Query

rows = g.query("""
    SELECT ?s ?label WHERE {
        ?s <http://www.w3.org/2000/01/rdf-schema#label> ?label
    } LIMIT 10
""")
for row in rows:                       # SELECT -> list of {var: Term}
    print(row["s"].value, row["label"].to_python())

g.query("ASK { ?s ?p ?o }")            # -> bool
g.query("CONSTRUCT { ... } WHERE { ... }")   # -> list of (s, p, o) Term triples

Every value is a Term with .kind ("iri" / "literal" / "bnode" / "triple" for RDF-star), .value, .datatype, .lang, plus:

  • .to_python() — int/float/bool for the common XSD datatypes, else the string
  • .n3 — the term back in N-Triples surface form

g.query_raw(q) returns the engine's JSON envelope unparsed (the same shape the playground uses), and g.query_df(q) returns a pandas DataFrame (pip install rete-graph[pandas]).

Why did a FILTER return nothing? g.last_warnings()

CONTAINS(?s, "x") on an IRI, STRSTARTS(?year, "15") on an integer and CONTAINS(?label, "x"@fr) against an @en label are SPARQL type errors. A FILTER treats a type error as false, so the query returns no rows and no error. The rows stay exactly as SPARQL defines them, and g.last_warnings() says why (the rule):

g.query('SELECT ?s WHERE { ?s rdfs:label ?l FILTER(CONTAINS(?s, "geneva")) }')  # []
g.last_warnings()
# [{'severity': 'type-error', 'function': 'CONTAINS', 'argument': 1, 'argKind': 'iri',
#   'count': 1, 'sample': '<http://ex.org/map/geneva-1572>',
#   'hint': "wrap it in STR() to match the IRI's text", 'message': 'CONTAINS received an IRI …'}]

It describes the most recent query() / query_raw() / query_df() on that graph, and is [] when there is nothing to report. query_raw() also carries the list as warnings when it is non-empty.

Run the file's own example queries

A .rete can carry example SPARQL queries inside the file (in its Dataset Card); CLI-built datasets ship a whole tiered starter library. g.examples() reads them — on a remote file that costs one small ranged read:

g = rete.open("https://data.graphplaza.com/boe/boe-2026-07-07.rete")
for ex in g.examples():                     # 20 entries on this dataset
    print(ex["title"], "—", ex["question"])
g.query(g.examples()[0]["sparql"])          # and they just run

Embed your own with Builder.example() (rich: title + question + SPARQL) or Builder.card(example_queries=[...]) (plain strings).

Reasoning

g.query(q, reason=True) turns on OWL 2 QL entailment — rdfs:subClassOf, subPropertyOf, domain/range and friends, computed by query rewriting over the file's ontology, so it works over lazy remote files without materializing anything. See Reasoning & coherence.

Union default graph

g.query(q, union_default_graph=True) (also query_raw and query_df) lets a pattern outside GRAPH match the default graph and every named graph, a triple stored in several graphs counting once. It is not standard SPARQL, so it is off by default; a FROM in the query still wins. See Union default graph for what it costs.

Federation

SPARQL 1.1 SERVICE works out of the box: join a .rete file against any public SPARQL endpoint (Wikidata, DBpedia, …) in one query. See Federated queries.

Build .rete files from Python

rete.build() produces a complete file image, ready to open(), save, or upload:

data = rete.build(nt_text)                   # N-Triples text ("nt", "nq", "ttl", "trig", "rdfxml", "jsonld", "yamlld")
pathlib.Path("out.rete").write_bytes(data)

It also accepts a graph object from another RDF library — anything with a .serialize(format=...) method (duck-typed; rdflib is not a dependency). An rdflib Graph round-trips as N-Triples; a context-aware Dataset / ConjunctiveGraph as N-Quads, so named graphs survive:

import rdflib, rete_graph as rete

g = rdflib.Graph()
g.parse("mydata.ttl")

rg = rete.open(rete.build(g))                # rdflib -> .rete -> SPARQL

For step-by-step preparation — an embedded Dataset Card, pyramid options, the full-text index — use the lazy Builder: configure, then run() and export():

builder = (
    rete.Builder()
    .add_file("people.ttl")
    .card(title="People", license="CC0-1.0")
    .pyramid(algo="louvain")        # or "types", or .pyramid(False)
    .text_index()
)
builder.run()                        # bytes; stats in builder.stats
builder.export("people.rete")

Add .derive_card() and the card also carries the auto-derived profile — predicates, classes, vocabularies, datatypes, languages, hubs, signals and the tiered starter-query library — from the same code rete build --card runs, so the same graph yields a byte-identical card. It is opt-in: derivation walks the graph twice more, and the default keeps writing exactly the bytes it always did.

Jelly

Jelly (.jelly) is a binary RDF format, so it goes in as bytes, not text: rete.build(open("data.jelly", "rb").read(), "jelly"). Builder.add_file does that for you and knows the .jelly suffix. A text argument with "jelly" is a ValueError.

RDF 1.2 and quoted triples

The same --quoted-triple-syntax vocabulary and defaults as the CLI (see Interop), in both directions.

Reading. In Turtle and TriG, << s p o >> is a quoted triple under RDF-star and a reifier under RDF 1.2 — the same bytes, two graphs — so the build calls take quoted_triple_syntax: "rdf-star" (the default, what every earlier release read) or "rdf12" (<<( s p o )>> triple terms, << … >> reifiers, {| … |} annotations, "…"@lang--dir literals). N-Triples, N-Quads and RDF/XML ignore it. "trig" is a format too, and .trig a known suffix.

g = rete.open(rete.build(ttl_text, "ttl", quoted_triple_syntax="rdf12"))
rete.Builder().add_file("dump.trig", quoted_triple_syntax="rdf12").run()

Writing. Graph.to_nquads() spells a quoted triple as the RDF 1.2 triple term <<( s p o )>> by default, which Oxigraph/pyoxigraph 0.5 and Jena 5 load; quoted_triple_syntax="rdf-star" writes rete's stored <<s p o>>. RDF 1.2 has no triple term in subject position, so such a statement raises ValueError under "rdf12". A graph with no quoted triples is written identically either way. (iter_quads() yields the stored tokens.)

The card carries signals.quoted_triples — {"present": …, "export_surfaces": […], "export_default": "rdf12"} — measured from the file's header on read, so files built before the signal existed answer it too.

The full walkthrough (every card field, pyramid trade-offs, verification) is in Python: build a .rete. In-memory builds suit tests and small-to-medium graphs (say, up to a few million triples). For big datasets use the rete build CLI, which streams from disk and compresses harder — see Tables, VKG & big builds.

Search and overview

g.schema()                    # {"classes": [(iri, count)], "relations": [(s, p, o, count)]}
g.prefix_search("Berl")       # label autocomplete -> [(label, subject_iri)]
g.text_search("volcano")      # full-text; needs a file built with --text-index
g.info()                      # {"quads": ..., "terms": ..., "pyramidLevels": ...}
g.quads, g.terms              # header counts, as properties
g.graph_names()               # named graphs in a dataset
g.content_hash()              # blake3-16 hex — a stable cache key

SHACL validation

From 0.2.2, validate a graph against SHACL Core shapes written in Turtle — data-quality contracts checked in place:

report = g.shacl("""
@prefix sh: <http://www.w3.org/ns/shacl#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

[] a sh:NodeShape ;
  sh:targetClass <https://example.org/Person> ;
  sh:property [ sh:path rdfs:label ; sh:minCount 1 ] .
""")
report["conforms"]            # True / False
report["results"]             # one entry per violation (focus node, path, message)
g.shacl(shapes, format="ttl") # the same report as sh:ValidationReport Turtle

Validation is lazy-aware: on a remote graph only the shapes' targets are fetched — the index is consulted in place, never the whole file (a broad sh:targetClass fetches many targets, so scope shapes tightly). graph="<iri>" validates one named graph instead of the default graph.

Browser Python: JupyterLite & marimo WASM

From version 0.2.0 the release also ships PyEmscripten (Pyodide) wheels, so the client runs inside browser Pythons — a JupyterLite site, marimo's WASM playground — with no server anywhere. Try it right now in the JupyterLite experiment bundled with these docs.

# JupyterLite / Pyodide 314 (Python 3.14) — needs rete-graph >= 0.2.1
%pip install rete-graph

import rete_graph as rete
g = rete.open("https://data.graphplaza.com/boe/boe-2026-07-07.rete")
g.query("SELECT ?s ?p ?o WHERE { ?s ?p ?o } LIMIT 5")

On the older Pyodide 0.29 (Python 3.13), whose installer predates the PEP 783 wheel-tag spelling PyPI requires, %pip install rete-graph finds no wheel. Each GitHub release after 0.3.6 carries the same build retagged for it, rete_graph-<version>-cp39-abi3-pyodide_2025_0_wasm32.whl:

  1. download that file from the release page;
  2. in JupyterLite, upload it with the file browser into the folder of your notebook;
  3. install it from the notebook's filesystem:
%pip install emfs:./rete_graph-<version>-cp39-abi3-pyodide_2025_0_wasm32.whl

The emfs: prefix makes the installer read the kernel's filesystem; a bare ./rete_graph-….whl is fetched as a URL and fails. In any other Pyodide 0.29 runtime (marimo's WASM notebooks, plain Pyodide), write the file to the in-browser filesystem and run await micropip.install("emfs:/path/to/rete_graph-….whl"). Installing by the release download URL does not work: GitHub serves release assets without CORS headers, so the browser blocks the request. Versions up to 0.3.6 stay installable by URL from our own CORS-enabled bucket, e.g. %pip install https://data.graphplaza.com/wheels/rete_graph-0.3.6-cp39-abi3-pyodide_2025_0_wasm32.whl; nothing newer is uploaded there.

Under the hood, remote reads use synchronous XMLHttpRequest range requests — allowed only in web workers, which is where JupyterLite and marimo run their kernels, so it just works there. The file's host must send CORS headers and honor Range, the same contract as the playground.

Differences from the native wheels, by browser necessity:

  • Range fetches are sequential (no threads in wasm) — fine in practice, since the block cache already coalesces reads.
  • SERVICE federation is unavailable.
  • In-browser build() writes uncompressed sections (like the playground's Build tab); every reader accepts them.
  • wasm32 means a 4 GiB memory ceiling — lazy remote querying is the intended use, not giant in-memory builds. (A wasm64 build lifts this, and is tracked as future work pending Pyodide support.)

Pyodide wheels are per-ABI-year (not abi3): current Pyodide releases are covered; if yours isn't, micropip.install("<url of the .whl>") works from any CORS-enabled host.

Guarantees and threading

  • No silent partial results: if a range fetch fails mid-query on a lazy handle, the query raises instead of returning fewer rows.
  • The GIL is released around every engine call — opening, query(), dumps and exports, searches, SHACL, build() and card derivation — so other Python threads keep running during long calls, and two threads can query or build at once.
  • Native wheels use every core inside those calls (since 0.4.2): a build's sorts, permutations and compression and a query's dictionary decodes run on rete-core's parallel feature (rayon), with the same bytes and rows as a single-threaded build. RAYON_NUM_THREADS=n caps the pool; set it before the first call. Remote range reads fan out over the HTTP reader's own threads (16-way, like the CLI and the playground). rete_graph.__engine_features__ lists parallel and http on a native wheel; the Pyodide wheels have no threads and list neither. As with any rayon user, the pool does not survive os.fork(): a child forked after its parent built or queried must not call into the engine (it can hang). With multiprocessing, use the spawn or forkserver start method.
  • Errors surface as ordinary exceptions: ValueError for bad SPARQL or RDF input, RuntimeError for I/O and format problems.

Feature matrix

CapabilityPythonNotes
SPARQL SELECT / ASK / CONSTRUCT / DESCRIBE✅query()
Lazy remote open (HTTP Range)✅open(url), custom headers
Lazy local open✅positional reads, no whole-file load
Custom reader objects✅open(reader=...) — fsspec/S3
OWL 2 QL reasoned queries✅query(..., reason=True)
Union default graph✅query(..., union_default_graph=True)
SERVICE federation✅host-injected HTTP client
Build from RDF text / rdflib objects✅build(), Builder
Dataset Card: embed + read back✅Builder.card(), Graph.card() (ranged on remote)
Dataset Card: derive the profile✅Builder.derive_card() — opt-in; identical to rete build --card
Example queries in the file✅Builder.example(), Graph.examples()
Build options: pyramid algo, text index✅Builder.pyramid()/text_index()/type_predicate()
Schema profile, prefix & text search✅schema(), prefix_search(), text_search()
pandas DataFrames✅query_df(), [pandas] extra
Browser Python (Pyodide)✅ 0.2.0JupyterLite / marimo WASM; no SERVICE, sequential fetches
SHACL validation✅ 0.2.2shacl() — lazy over shape targets
Reachability / communities / provenance (why)⏳planned bindings
Multi-shard federated open⏳planned
Writes / SPARQL UPDATE—.rete is immutable; use rete serve

For contributors

This page is for using the package. Building from source, the CI layout, the release/publishing procedure, and the checklist for adding new language clients live in Client development & releases.