@lde/distribution-health

Derives a distribution’s usability from two separately-produced signals:

• reachability – can the distribution be fetched? (HTTP/SPARQL level, produced continuously by the crawler’s probe)
• validity – does the fetched content actually parse as RDF? (a by-product of parsing: shallow by the crawler, deep by the knowledge-graph pipeline)

This is a pure leaf: it interprets the raw results of @lde/distribution-probe and @lde/sparql-importer rather than producing them. It contains no I/O and returns plain TypeScript – converting a verdict to RDF is the consumer’s job (see below), so the package stays vocabulary-agnostic.

Installation

npm install @lde/distribution-health

What it provides

• A ValidityVerdict type with a typed failure reason (parse-error / empty), a best-effort parser message, the validatedFingerprint it was judged against, and its producer depth (shallow / deep).
• Mappers from a probe result (shallow) and an import outcome (deep) to a verdict.
• The usability rollup: (reachability, validity verdicts) → { usable | unusable | unknown, cause }, the one canonical rule consumers share. Reachability dominates; a stale verdict (one whose fingerprint no longer matches the currently-observed one) decays to unknown; a deep verdict beats a shallow one.

API

function usability(
  reachability: Reachability,
  verdicts: readonly ValidityVerdict[],
): Usability;

function probeResultToVerdict(
  result: ProbeResultType, // from @lde/distribution-probe
  validatedFingerprint: string | null,
): ValidityVerdict | null;

function importOutcomeToVerdict(
  outcome: ImportFailed | ImportSuccessful, // from @lde/sparql-importer
  validatedFingerprint: string | null,
): ValidityVerdict;

Supporting types: Reachability ({ reachable: boolean, fingerprint: string | null }), Usability ({ state: UsabilityState, cause?: UsabilityCause, shallow?: true }), UsabilityState ('usable' | 'unusable' | 'unknown'), UsabilityCause ('invalid' | 'unreachable' | 'no-verdict' | 'stale-verdict'), and ValidityDepth ('shallow' | 'deep').

probeResultToVerdict() – the shallow producer

Returns null when the probe carries no validity signal: a SPARQL probe result or NetworkError (only data-dump probes validate bodies), a network or HTTP-level failure, or a body the probe did not parse. Otherwise:

• a failureReason of ‘Distribution is empty’ or ‘Distribution contains no RDF triples’ maps to valid: false, reason: 'empty'; any other failureReason maps to valid: false, reason: 'parse-error' with the reason as message;
• valid: true requires positive evidence that the probe parsed the body: the content type must be an RDF serialization the probe parse-validates and the contentSize must be unknown (null) or at most 10 240 bytes.

importOutcomeToVerdict() – the deep producer

An ImportFailed is valid: false, reason: 'parse-error' with the import error as message. An ImportSuccessful is valid: true – except when its tripleCount === 0, which is judged valid: false, reason: 'empty': a dump that parses to nothing is faulty, not usable.

usability() – the rollup

• Unreachable → { state: 'unusable', cause: 'unreachable' }, regardless of verdicts.
• Reachable, no applicable verdict → { state: 'unknown' } with cause 'no-verdict' (never judged) or 'stale-verdict' (judged, but no verdict is fresh anymore).
• Reachable with a fresh verdict → usable, or { state: 'unusable', cause: 'invalid' }. A deep verdict beats a shallow one; when the state rests on a shallow verdict only, Usability.shallow is true so consumers can mark it as not yet deeply confirmed.

Freshness gate: a verdict applies only when its validatedFingerprint equals the reachability check’s current fingerprint and both are non-null – a null fingerprint never compares equal, so an unfingerprintable source is never fresh and its verdicts decay to unknown.

RDF is the consumer’s responsibility

This package emits no RDF and coins no vocabulary, mirroring @lde/iiif-validator (which returns a TS verdict that the NDE knowledge-graph pipeline maps to RDF). A consumer turns a ValidityVerdict into its own DQV/PROV quads under its own namespace – e.g. NDE writes a dqv:QualityMeasurement on the distribution under def.nde.nl. The ValidityFailureReason local names are chosen to drop straight into a SKOS failure scheme (<scheme>#${reason}) with no lookup table.

The normative usability rule and the rejected alternatives are recorded in the PRD: netwerk-digitaal-erfgoed/dataset-register#2103.