thisisneat 0.2.2

Knowledge graph transformation

kNowlEdge grAph Transformer (NEAT)

NEAT is a domain expert centric and developer friendly solution for rapid:

• data modeling
• extraction, transformation and loading of instances
• and ingestion of the models and instances (i.e. knowledge graphs) into Cognite Data Fusion

NEAT is using open and globally recognized standards maintained by the World Wide Web Consortium (W3C). NEAT represents an essential tool for creation of standardized, machine-actionable, linked and semantic (meta)data.

NEAT is a funny acronym derived from kNowlEdge grAph Transformer produced using ACRONIMIFY.

History

NEAT emerged from years of experience in semantic tooling and knowledge graph development. The foundation was laid in 2020 with sheet2rdf, an Excel-based tool that trained data stewards to build domain-specific knowledge graphs and supported CI/CD processes in the Dutch Covid program and european wind energy community.

By mid of 2022, sheet2rdf was used in several POCs in Cognite. As Cognite's Data Modeling Service (DMS) development progressed, the need for simplified data modeling experience led to demonstration of proto-NEAT, known as sheet2fdm, an extension of sheet2rdf, enabling semantic data model definitions in OWL, SHACL, Python and GraphQL (see e.g., wind energy data model) using a simplified version of sheet2rdf Excel template.

Presented in various forums in 2022, this approach paved the way for NEAT’s formal development in November 2022 to enable cost-saving and empowerment of Cognite customers to self-sufficiently maintain and onboard knowledge graphs to Cognite Data Fusion.

Authorship

Authors

The plot below shows the NEAT authorship from the start until present day.

Current authors

• Nikola Vasiljević
• Anders Albert
• Rogerio Júnior

Former authors

• Aleksandrs Livincovs
• Julia Graham

Contributors

We are very grateful for the contributions made by:

• Marie Solvik Lepoutre, who improved RDF triples projections to Cognite Data Fusion
• Bård Henning Tvedt, who implemented IMF importer
• Hassan Gomaa, who extended the DEXPI extractor
• Kristina Tomičić, who implemented Data Model and Instances visualization

Sponsors

NEAT is developed and maintained by Cognite. We are grateful for the past support of our sponsors, who funded us to develop NEAT and to make it open source.

• Statnett - the MIMIR team (Ola Hagen Øyan, Olav Westeng Alstad,Andreas Kimsås and Anders Willersrud) – who supported the development of NEAT from end of 2022 to mid of 2023 and its integration with Statnett's infrastructure, where NEAT was battle-tested as a tool for non-sematic experts to define data models and transform large knowledge graphs representing the entire Norwegian power grid system. Without Statnett's support NEAT would not exist in the first place, and would not be open-source.
• Aker Solutions – the IMod Team (currently lead by Maria Kenderkova), who funded development of NEAT from mid of 2023 till end of 2024 (multi-level and role-based data modeling, support for ontologies, IMF, DEXPI, AML,...) , as well who were early adopters and embraced NEAT and who were patient with us when things did not work so well. Aker Solutions was instrumental to elevating NEAT to a product level, and who selflessly advocate for NEAT globally.

Installation

pip install cognite-neat

Usage

The user interface for NEAT is a notebook-based environment. Once you have set up your notebook environment, you start by creating a CogniteClient and instantiate a NeatSession object.

from thisisneat import NeatSession, get_cognite_client

client = get_cognite_client(".env")

neat = NeatSession(client)

neat.read.cdf.data_model(("my_space", "MyDataModel", "v1"))

Documentation

For more information, see the documentation