quadipy

quadipy is a python package to help transform structured data into RDF graph format.

We built quadipy to enable developers to build a config based ingestion pipeline to an RDF data store (think like FiveTran or Stitch but for RDF). quadipy won't explcitly handle connections to different systems but will allow you to configure the RDF data you want to create from any data source. quadipy leverages RDFLib to pythonically structure RDF data.

The goal with this project is to enable transforming any tabular data structure into graph based RDF data. We go into depth here on how we have used this config based system to build out our internal knowledge graph at Vouch! You can also check out our talk at KGC'22 on YouTube: Modeling the startup ecosystem using a config based knowledge graph.

An example below shows what we mean by translating some tabular data into RDF graph data.

For a step by step walk-through of how to do this with more examples, visit tutorials

Dev Setup

Run to set up your dev enviornment

make setup

Usage

from quadipy import GraphFormatConfig

config = GraphFormatConfig.parse_file("path/to/config.json")
quads = [config.quadify(record) for record in records] # records is an Iterator[Dict]

quadipy can work with a variety of different data sources as long as what is sent to the quadify method is a Dict

Each Quad created has a .to_tuple() method that converts it to a tuple to help facilitate working with RDFLib Graphs

from rdflib import URIRef, Graph
from quadipy import Quad

quad = Quad(subject=URIRef("Alice"), predicate=URIRef("knows"), obj=URIRef("Bob"), graph=None)
g = Graph()
g.add(quad.to_tuple())

Setting up a GraphFormatConfig

The main value of quadipy comes from the GraphFormatConfig, which takes in a few parameters to configure the transformation of your data into RDF graph format. We provide configuration examples in the examples directory to help you get started. The full list of fields that can be configured is described below:

Validate Config files

To make sure the config files are valid, run the CLI by using the command

quadipy validate {path}

where path could be the directory for all the config files, e.g. examples or a single config file e.g. examples/simple.json

This script uses pydantic validator to make sure the config file is a valid JSON file, the required fields are presented and the predicate_uris are valid URIs ("valid" defined by RDFLib here)