@lde/pipeline-shacl-validator

SHACL validation for @lde/pipeline.

Validates RDF quads produced by pipeline stages against SHACL shapes, streaming the per-dataset SHACL validation report to any number of configured Writers. Shapes can be provided in any RDF serialization (Turtle, JSON-LD, N-Triples etc.).

Installation

npm install @lde/pipeline-shacl-validator

Usage

import {
  Pipeline,
  Stage,
  SparqlConstructReader,
  FileWriter,
  SparqlUpdateWriter,
} from '@lde/pipeline';
import { ShaclValidator } from '@lde/pipeline-shacl-validator';

const validator = new ShaclValidator({
  shapesFile: './shapes.ttl',
  reportWriters: [
    new FileWriter({ outputDir: './validation', format: 'turtle' }),
    new SparqlUpdateWriter({ endpoint: new URL('http://store/update') }),
  ],
});

const pipeline = new Pipeline({
  // ...
  stages: [
    new Stage({
      name: 'transform',
      readers: new SparqlConstructReader({ query: '...' }),
      validation: {
        validator,
        onInvalid: 'write', // 'write' | 'skip' | 'halt'
      },
    }),
  ],
});

await pipeline.run();

Severity and conformance (conformanceDisallows)

By default, a result of any severity (sh:Violation, sh:Warning or sh:Info) makes validation fail. This matches the SHACL 1.2 default conformance-disallow set.

To fail on selected severities only (e.g. treat warnings and info as advisory, as SHOULD-level findings usually are) pass the set of severity IRIs that disallow conformance:

import { ShaclValidator, severity } from '@lde/pipeline-shacl-validator';

const validator = new ShaclValidator({
  shapesFile: './shapes.ttl',
  conformanceDisallows: [severity.violation],
});

Results outside the set are still reported; they just no longer flip conforms to false or count towards violations. Custom severity IRIs (allowed by SHACL 1.2) work too.

When the option is set, the written report stays self-describing per SHACL 1.2: its sh:conforms reflects the configured set, and the set itself is declared on the report via sh:conformanceDisallows.

onInvalid options

Value	Behaviour
'write'	Write quads to the output even if validation fails (default)
'skip'	Discard invalid quads silently
'halt'	Throw an error, stopping the pipeline

Report writers

Each validate() call that produces validation results (of any severity, whether or not they fail conformance) fans the SHACL report quads (sh:ValidationResult triples, etc.) out to every configured reportWriter. The validator opens one long-lived run per writer via Writer.openRun() – with a fabricated run context, since the report stream is the validator’s own run rather than the pipeline’s – and streams each batch to it via RunWriter.write(dataset, quads). Each run’s RunWriter.flush(dataset, 'success') is invoked from ShaclValidator.report(dataset) – i.e. once the pipeline finishes a dataset.

When results were written, validate() returns a message of the form Report sent to N writer(s); with onInvalid: 'halt' the pipeline surfaces this message in the halt error, pointing at where the report landed.

A validate() call with an empty quad batch short-circuits to { conforms: true, violations: 0 } without touching the SHACL engine.

A dataset validated with no results of any severity still yields a report: a minimal conforming sh:ValidationReport (with sh:conforms true and no results) is written from report(dataset). So every validated dataset produces a report, and its absence means the dataset was not validated – a distinction consumers can rely on. It also means a re-run in which a dataset becomes conforming overwrites its prior report rather than leaving stale violations behind. Only a dataset that was never validated (no validate() call reached it) produces nothing.

Validators with no reportWriters only produce aggregate counts ({ conforms, violations, quadsValidated }); the report quads themselves are discarded. This is deliberate – callers who only need pass/fail metrics don't have to wire up a sink – but it does mean misconfiguring (passing reportWriters: [] while expecting persistence) silently loses violation detail. Configure at least one writer in production pipelines.

The bundled FileWriter and SparqlUpdateWriter already implement the Writer contract; bring your own for custom destinations.

For convenience, the package re-exports the SerializationFormat type (and the Validator, ValidationReport and ValidationResult types) from @lde/pipeline.

Blank-node-free reports

shacl-engine emits the sh:ValidationReport, every sh:ValidationResult and any anonymous sh:sourceShape as blank nodes. Before writing, ShaclValidator rewrites each one to a dataset-scoped IRI of the form <dataset>/.well-known/shacl#<batch>-<label>. This keeps a file-based served store (e.g. the Dataset Knowledge Graph) from fusing one dataset's results into another's when it cats every per-dataset n-quads file into a single index – blank-node labels are only document-scoped and recur across files (see ldelements/lde#478). The dataset IRI rules out fusion across datasets; <batch>, a hash of the report's quads, rules out fusion across the separate validate() batches that land in one dataset's validation graph.

Filesystem collisions with FileWriter

FileWriter derives its filename from dataset.iri only. If the pipeline's main writer and a report writer both target the same outputDir with the same format, they will collide on the same path and the second open will truncate the first. Use a separate outputDir for validation reports:

new ShaclValidator({
  shapesFile,
  reportWriters: [new FileWriter({ outputDir: './output/validation' })],
});

Named graphs with SparqlUpdateWriter

SparqlUpdateWriter defaults to dataset.iri.toString() as the named graph URI. A report writer that shares the endpoint with the pipeline's main writer would otherwise land the SHACL report in the same graph as the dataset's data – and CLEAR GRAPH on first write per dataset would erase it. To keep validation results in a separate graph, pass graphIri to derive the target graph from the dataset:

new SparqlUpdateWriter({
  endpoint,
  auth,
  graphIri: (dataset) =>
    new URL(
      `https://example.org/shacl-validation/${encodeURIComponent(dataset.iri.toString())}`,
    ),
});