rdflib field notes: ns.format matches nothing, and parse() goes online

Sat, 11 Mar 2023 12:37:00 +0100

UUpdate 2026-11-08: retested on rdflib 7.6.0 and added the ConjunctiveGraph deprecation, pyoxigraph, and the RDF 1.2 and SPARQL 1.2 status.

This lookup returns an empty list, with no exception and no warning:

from rdflib import Graph, Literal, Namespace, URIRef

DCT = Namespace("http://purl.org/dc/terms/")
doc = URIRef("http://example.org/doc")

g = Graph()
g.add((doc, DCT["format"], Literal("text/csv")))

list(g.objects(doc, DCT.format)) # []
list(g.objects(doc, DCT["format"])) # [rdflib.term.Literal('text/csv')]

Namespace subclasses str, so DCT.format is the bound method str.format, and a lookup on it matches nothing. RDF code tends to fail this way: you get fewer triples, and that looks exactly like sparse data.

I collected these while re-reading a crawler I wrote in early 2023, and rebuilt each as a minimal repro on rdflib 6.3.2 and 7.6.0 under Python 3.11. Both versions behave the same unless noted. They show what the code does. I have no production logs that say what any of them cost.

Use `ns["format"]` for names that `str` already owns

format, count, index, join, and every other str method name resolve to the method. title is the exception, special-cased to return a URIRef. Writes are safer than reads here: g.add() raises AssertionError on the method object.

For published vocabularies, import from rdflib.namespace. DCTERMS.format is a real URIRef and DCTERMS.nonsense raises AttributeError. For your own vocabulary, ClosedNamespace rejects unknown terms, but .format on it is still the str method, so keep the brackets.

A typo in the base IRI is still a valid IRI

NS = Namespace("https://example.org/vocab/mediaTyp")
NS.mediaType # https://example.org/vocab/mediaTypmediaType

My crawler shipped this. The branch using it asked for a predicate no document contains. ClosedNamespace produces the same glued IRI, because it validates the term and has no opinion on the base. A one-line test covers it, with an allowlist for the rare namespace that ends differently:

assert str(NS)[-1] in "#/"

A failed parse keeps what it already read

Feed Graph.parse() five good N-Triples lines and one bad one. It raises ParserError, and len(g) is 5. Turtle does the same with BadSyntax. If you parse into a shared graph, a truncated download leaves half a document behind.

Give each document its own store and merge only after the parse succeeds:

import tempfile
from pathlib import Path
from rdflib import Graph, URIRef, plugin
from rdflib.store import Store

def load_document(source, fmt, target):
 with tempfile.TemporaryDirectory() as tmp:
 path = str(Path(tmp) / "graph")
 store = plugin.get("LevelDB", Store)(identifier=URIRef(path))
 graph = Graph(store, identifier=URIRef("urn:doc"))
 graph.open(path, create=True)
 try:
 graph.parse(source, format=fmt)
 target += graph
 finally:
 graph.close()

With a bad document, this raises an exception and target stays empty. The disk store also caps memory. Loading 200,000 synthetic triples (14 MB of N-Triples, one run on Apple Silicon, rdflib 6.3.2, 30 MB baseline after imports):

Store	Time	Peak RSS	Disk
rdflib `Memory`	2.6 s	351 MB	0
`rdflib-leveldb`	15.3 s	85 MB	39 MB
pyoxigraph 0.5.13 `bulk_load`	0.5 s	200 MB	10 MB

Read that as a rough shape and benchmark your own data. rdflib-leveldb isn’t on PyPI, so install it from git. Its last commit is from December 2021, though the snippet above still passes on 7.6.0. On macOS, I used plyvel-ci because plyvel has no Mac wheels. Pyoxigraph has its own API and isn’t a drop-in rdflib store.

A disk store bounds the triples and leaves the input alone. The Turtle parser calls stream.read() on the whole source, with the comment # Not ideal.

`parse()` can go online

Parsing JSON-LD fetches remote @context values. The RDFLib security page says so. Direct cycles are caught (recursive context inclusion after 2 requests). A chain of contexts that each point to a new one isn’t: with Python’s default recursion limit, one small document cost 967 requests to my local server before RecursionError.

Block the network during the parse step:

import urllib.error, urllib.request

class NoNetwork(urllib.request.BaseHandler):
 handler_order = 100 # before the stock HTTPHandler (500)

 def http_open(self, req):
 raise urllib.error.URLError(f"blocked: {req.full_url}")

 https_open = http_open

urllib.request.install_opener(urllib.request.build_opener(NoNetwork()))

Leave out handler_order and the stock handler answers first, so the request goes through. My first attempt did exactly that. The opener is process-global, and the RDFLib docs rank OS-level isolation such as containers above Python-level controls, so use it in addition to those on untrusted input.

Then make the following links a separate, bounded step that you control:

from collections import deque

def crawl(seed, fetch, max_depth=3):
 seen, queue = {seed}, deque([(seed, 0)])
 while queue:
 iri, depth = queue.popleft()
 references = fetch(iri) # fetch, parse in isolation, return IRIs to follow
 if depth < max_depth:
 for ref in references:
 if ref not in seen:
 seen.add(ref)
 queue.append((ref, depth + 1))

On a graph with a cycle and a chain, this visits each IRI once and stops at depth 3.

Keep a workaround’s reason in a test

rdflib #1195 reported that a CONSTRUCT response with application/rdf+xml; charset=UTF-8 broke the plugin lookup. My code registered that literal string as a parser name and pointed a comment at the issue.

Against a fake endpoint sending that header, rdflib 5.0.0 raises PluginException and 6.0.0 works without the workaround (both on Python 3.9). My pin was already 6.0.0 in the commit that added it, so on this path it never changed, and it stayed that way through 6.3.2. Watching the issue wouldn’t have helped: it closed in August 2022 on a PR that only added tests.

A test that exercises the upstream behavior with the workaround off tells you the day it can go. plugin.get() itself still rejects that string on 7.6.0, so strip parameters if you call it directly.

`Dataset` isn’t a drop-in for `ConjunctiveGraph`

ConjunctiveGraph raises a DeprecationWarning on 7.6.0 and the replacement is Dataset, which has a different default. With one quad in a named graph:

Class	`triples()`	`SELECT ?s ?p ?o`
`ConjunctiveGraph()`	1	1
`Dataset()`	0	0
`Dataset(default_union=True)`	1	1

Migrate with default_union=True, unless you want queries limited to the default graph.

On standards, as of November 2026: RDF 1.2 Concepts is a Candidate Recommendation and SPARQL 1.2 Query is a Working Draft, so SPARQL 1.1 remains the Recommendation.

Takeaway

Most of these bugs arrive as an empty result, and nothing downstream complains. For each predicate your pipeline depends on, keep a fixture with one known triple and assert that your code finds it.

Start with this search for unbracketed str names in your own code:

grep -rnE '\b[A-Za-z_]+\.(format|count|index)\b([^(]|$)' --include='*.py' .

Json-Ld on Lessons learned