From SQL Constraints to SHACL Shapes: Declarative Data Validation

SHACL is the way!

Part 4 of my From Data Engineering to Knowledge Engineering series is all about data quality and data validation!

Generated with the help of GPT 5.2.

Imagine deploying your planetary data pipeline to production. Everything looks great until an analyst discovers that Aitne, a tiny moon of Jupiter, somehow has an albedo of 4.0 (physically impossible), orbits both Jupiter AND Saturn (also impossible), and is larger than Jupiter itself (definitely impossible). Your database let all of this through because cross-table validation logic was scattered across three different systems, none of which talked to each other. This is the daily reality for data engineers working with SQL. Let us dive into the world of validating data as a knowledge engineer!

Setting the scene: the data!

I am going to continue with my data about planets and their satellites, within our own Solar System.

┌─────────────────────────────┐
│         PLANETS             │
├─────────────────────────────┤
│ PK planet_id (SERIAL)       │
│    name (VARCHAR) UNIQUE    │
│    length_of_day (DECIMAL)  │
│    mean_temperature (BIGINT)│
│    orbital_period (DECIMAL) │
└─────────────────────────────┘
              △
              │
              │ orbits (FK)
              │
┌─────────────────────────────┐
│    NATURAL_SATELLITES       │
├─────────────────────────────┤
│ PK satellite_id (SERIAL)    │
│    name (VARCHAR) UNIQUE    │
│    albedo (DECIMAL)         │
│    radius (DECIMAL)         │
│ FK orbits_planet_id (INT)   │
└─────────────────────────────┘

Regard this as the data model for your SQL-based database. For a knowledge graph database, it’s the ontology that is the database schema/model. It lives together with the data in its graph structure. Some key differences between relational databases and knowledge graphs:

We saw examples of several relational query pains in my previous article SPARQL for SQL Developers: A Translation Guide, in this article I will focus on the data quality.

Traditional Data Quality Assessment

Let’s look at a few different approaches to validate data stored in databases, first using traditional approached such as database constraints for SQL, dbt Tests and Great Expectations (Python). I’ll highlight some limitations of each approach, before introducing data validation for knowledge graphs in the next section.

Things we want to say about planets:

• name as string,

• length_of_day is a mandatory decimal above 0.0,

• mean_temperature is a mandatory long value,

• orbital_period is a decimal above 0.0

Things we want to say about natural satellites:

• name as string,

• albedo double value between 0 and 1,

• that a satellite orbits exactly one planet,

• radius some double value above 0.0

Common for all of the traditional approaches is that cross-dataset validation is awkward, it requires joining/fetching data. The validation rules and constraints are not portable and do not travel with the data. Semantic constraints are difficult to express, try ”a natural satellite must orbit exactly one celestial body”. And there are no single source of truth; validation logic, schema and documentation are all separate.

Database Constraints (SQL/Relational)

CREATE TABLE planets (
  name VARCHAR(100) PRIMARY KEY,
  length_of_day DECIMAL CHECK (length_of_day > 0),
  mean_temperature BIGINT,
  orbital_period DECIMAL CHECK (orbital_period > 0)
);

CREATE TABLE natural_satellites (
  name VARCHAR(100) PRIMARY KEY,
  albedo DECIMAL CHECK (albedo >= 0 AND albedo <= 1),
  orbits VARCHAR(100) REFERENCES planets(name),
  radius DECIMAL CHECK (radius > 0)
);

Limitations with this approach is that

• it works only within one database,

• can’t express ”all satellites must orbit something”,

• hard to version control alongside data, and

• doesn’t travel with the data when exported.

dbt Tests (Modern Data Stack)

# schema.yml
models:
  - name: planets
    columns:
      - name: orbital_period
        tests:
          - not_null
          - dbt_utils.expression_is_true:
              expression: "> 0"
      
      - name: mean_temperature
        tests:
          - not_null

  - name: natural_satellites
    columns:
      - name: albedo
        tests:
          - dbt_utils.accepted_range:
              min_value: 0
              max_value: 1
      
      - name: orbits
        tests:
          - relationships:
              to: ref('planets')
              field: name

Limitations with this approach is that

• it is tied to dbt/SQL ecosystem,

• tests are separate from the data model,

• can’t express complex semantic rules easily,

• each test runs as a separate SQL query (performance)

Great Expectations (Python based)

import great_expectations as gx

# THIS IS A SNIPPET OF A FEW 
# VALIDATION POINTS TO REDUCE
# SPACE :-)

# Validate satellites
satellite_suite = gx.ExpectationSuite("satellites")
satellite_suite.add_expectation(
  gx.expectations.ExpectColumnValuesToBeBetween(
    column="albedo",
    min_value=0,
    max_value=1
  )
)
# Cross-dataset validation - more complex
satellite_suite.add_expectation(
  gx.expectations.ExpectColumnValuesToBeInSet(
    column="orbits",
    value_set=get_valid_planets()  # Need to fetch this
  )
)

Limitations with this approach is that

• it requires Python runtime,

• cross-dataset validation requires orchestration,

• expectations don’t describe the data model itself

• validation logic separated from schema

Knowledge Graph Validation

Everything in a knowledge graph is built up by the graph pattern of RDF. For the rest of the article, you will get exposed to a syntax for knowledge graphs called RDF Turtle. Resource Description Framework (RDF) is the foundational standard in the knowledge graph stack, and RDF Turtle is its most used syntax. If you need more insight on this, please have a look at my Substack article A brief introduction to the syntax of knowledge graphs.

Anyhow, I will repeat the two top things you need to know about knowledge graphs:

1. We talk in triples subject predicate object.

2. We use global, unique identifiers (IRIs) for all things.

A knowledge graph can store data, schema, constraints and rules in the same graph. It’s all triples!

The language for validating knowledge graphs is called the Shapes Constraint Language, and provides re-usable resources (pre-defined vocabulary) for describing constraints about our data. It’s expressed in RDF.

Example SHACL shapes (snippet). Woop, woop! See how we re-use the :nameShape. :-)

How to validate your knowledge graph

Let’s keep this simple and try validate some data that walks with its ontology (schema).

Pssst!! The data snippet here is the same as generated in my previous articles on From Data Engineering to Knowledge Engineering.

• Part 1: From Data Engineering to Knowledge Engineering in the blink of an eye

• Part 2: Data Engineering Ontologies

• Part 3: SPARQL for SQL Developers: A Translation Guide

• Part 4: this

A few lines of Python is all you need to spin up a SHACL Engine.

After running SHACL validation through a SHACL Engine (maplib used in this example), you get a validation report. If everything went well, the only content in such a report is:

_:1 a sh:ValidationReport ;
  sh:conforms true .

If the report conforms false, you will have a detailed description on exactly what went wrong! Let say if we have a moon with albedo of 4.0. The report would have looked something like this:

_:2 a sh:ValidationResult ;
  sh:resultSeverity sh:Warning ; 
  sh:sourceConstraintComponent sh:MaxInclusiveConstraintComponent ;
  sh:sourceShape :albedoShape ;
  sh:focusNode :Nereid ;
  sh:value "4.0"^^xsd:double ;
  sh:resultPath :albedo ;
  sh:resultMessage "Albedo must be between 0 and 1" ;

This result is in itself are machine-readable RDF triples that you can use for automatic decision making processes in your data pipelines. And it can travel with your data! After all--it’s all triples!

Practical Examples: The SHACL Way!

Here follows a few selected examples of typical data quality problems one can encounter with relational databased that are a no-brainer in knowledge graphs.

Orphaned Foreign Keys

SQL Problem

-- Deleting Jupiter leaves the satellite Aitne orphaned
DELETE FROM planets WHERE name = 'Jupiter';

SQL Solution

-- Foreign key with RESTRICT
FOREIGN KEY (orbits_planet_id) REFERENCES planets(planet_id)
    ON DELETE RESTRICT;
-- But this only prevents deletion, doesn't validate existing data

SHACL Solution

:SatelliteShape a sh:NodeShape ;
  sh:targetClass :NaturalSatellite ;
  sh:property :orbitsShape .

orbitsShape a sh:PropertyShape ;
  sh:path dt:orbits ;
  sh:minCount 1 ;
  sh:maxCount 1 ;
  sh:class :Planet .  # Must exist and be right type

Key difference: SHACL validates that the relationship exists AND that the target is the right type. SQL only checks that the ID exists.

Datatype Mismatches

SQL Problem

-- Temperature stored as VARCHAR
mean_temperature VARCHAR(50)  -- Could be "-200", "-200°C", "cold"

SQL Solution

-- Application-level validation or complex CHECK constraints
CHECK (mean_temperature ~ '^-?[0-9]+$')  -- Regex for numbers only

SHACL Solution

:PlanetShape a sh:NodeShape ;
  sh:targetClass :Planet ;
  sh:property :meanTempShape .

:meanTempShape a sh:PropertyShape ;
  sh:path :mean_temperature ;
  sh:datatype xsd:long ;  # Enforces numeric type
  sh:minCount 1 ;
  sh:maxCount 1 .

Key differences: SHACL enforces XSD datatypes natively. SQL requires regex or parsing.

Range Violations

SQL Problem

-- Albedo of 4.0 (impossible - should be 0-1)
INSERT INTO natural_satellites VALUES ('BadMoon', 4.0, 'Saturn', 25);

SQL “Solution”

albedo DECIMAL(4,3) CHECK (albedo BETWEEN 0 AND 1)
-- One constraint per column, per table

SHACL Solution

:SatelliteShape a sh:NodeShape ;
  sh:targetClass :NaturalSatellite ;
  sh:property :albedoShape .

:albedoShape a sh:PropertyShape ;
  sh:path :albedo ;
  sh:datatype xsd:double ;
  sh:minInclusive 0.0 ;
  sh:maxInclusive 1.0 .

Key differences: SHACL constraints are centralised in shapes, not scattered across table definitions.

Missing Required Data

SQL Problem

-- Planet without length_of_day
INSERT INTO planets (name, orbital_period) 
VALUES ('Pluto', 90560);  -- Missing length_of_day

SQL “Solution”

length_of_day DECIMAL(10,2) NOT NULL
-- But NULL vs. missing value semantics are messy

SHACL Solution

:PlanetShape a sh:NodeShape ;
  sh:targetClass :Planet ;
  sh:property :lengthOfDayShape .

:lengthOfDayShape
  sh:path :length_of_day ;
  sh:minCount 1 ;  # Must exist
  sh:datatype xsd:double .

Key difference: SHACL explicitly validates cardinality, separate from datatype.

Cross-Table Logic

SQL Problem

-- Moon bigger than its planet
UPDATE natural_satellites SET radius = 70000 WHERE name = 'Aitne';
-- Aitne is now bigger than Jupiter!

SQL Solution

-- Complex trigger or scheduled validation job
CREATE TRIGGER check_satellite_size
BEFORE INSERT OR UPDATE ON natural_satellites
FOR EACH ROW
EXECUTE FUNCTION validate_satellite_radius();
-- Requires procedural code

SHACL Solution

:SatelliteRadiusShape a sh:NodeShape ;
  sh:targetClass :NaturalSatellite ;
  sh:sparql [
    sh:message "Satellite cannot be larger than what it orbits" ;
    sh:select """
      PREFIX : <http://example.org/data/>
      SELECT $this ?planet
      WHERE {
        $this :radius ?satRadius ;
          :orbits ?planet .
          ?planet :radius ?planetRadius .
        FILTER (?satRadius > ?planetRadius)
      }
    """ ;
  ] .

Key difference: SHACL uses SPARQL for complex cross-entity validation. SQL requires triggers or application logic.

Summary

OK, I will leave it as this. In this article we have walked through some traditional methods for data validation; in SQL, dbt and Python. Then we have introduced and demonstrated SHACL, and how there, in a knowledge graph, is no syntactic difference between ontology (schema), shapes (validation constraints) and data;

It’s all triples! It all travels together. ❤️

We have had a look at some concrete data quality issues, with its solution in SQL vs. SHACL.

We can conclude with that SQL developers may end up with a Frankenstein monster of:

• Database constraints (basic stuff)

• dbt tests (run nightly, find issues late)

• Great Expectations (another tool to learn/maintain)

• Custom Python scripts (for weird edge cases)

• Airflow DAG with validation steps

• Documentation in Confluence (outdated)

• Slack alerts when things break

All to answer: “Is this data valid?”

All you need in life is a bit of SHACL. ❤️

Resources on SHACL

From the author

• SHACL for the Practitioner (2025), paperback and pdf

  • Order the book

  • Substack publication

• Co-author of SHACL Wiki together with Ivo Velitchkov.

• Co-author of Using the Shapes Constraint Language for modelling regulatory requirements together with Kristian Torkelsen (former NMA).

• SHACL Masterclass

Other hubs of knowledge

• Jose E. Labra Gayo, Eric Prud’hommeaux, Iovka Boneva, Dimitris Kontokostas, Validating RDF Data (2018)

• SHACL Wiki

• Shapes Constraint Language (W3C Spec)

Playing around

• SHACL Play!

• SHACL Playground by Zazuko

• The World’s Fastest Engine: maplib¹

From Data Engineering to Knowledge Engineering article-series

Part 1: From Data Engineering to Knowledge Engineering in the blink of an eye
Part 2: Data Engineering Ontologies
Part 3: SPARQL for SQL Developers: A Translation Guide
Part 4: this

1. https://github.com/oeg-upm/ERA-SHACL-Benchmark