kganalytica 0.1.12

Knowledge graph construction, comparison, and similarity toolkit for LLM contextual understanding

kganalytica

kganalytica is a Python research library designed for constructing, analyzing, and comparing knowledge graphs generated from natural language text.

The library enables researchers and developers to transform unstructured text such as paragraphs, documents, or model outputs into structured knowledge graphs (KGs) represented as triplets:

subject → relation → object

By converting textual information into graph structures, the library allows deeper analysis of factual information and relationships contained within the text.

---

Why This Library Is Important

Modern large language models generate answers in natural language, but evaluating whether those answers faithfully represent the underlying facts is challenging.

Traditional evaluation methods rely on:

• surface-level text similarity
• token overlap metrics
• embedding similarity

These methods often fail to capture whether the factual relationships are correct.

Knowledge graphs provide a structured representation of information where:

• entities represent real-world objects
• relations represent factual connections between entities

By constructing knowledge graphs from text, it becomes possible to compare facts rather than sentences.

---

What This Library Enables

Using kganalytica, a paragraph can be converted into a knowledge graph containing multiple factual triplets.

Example:

Input paragraph:

Albert Einstein was born in Germany.
He developed the theory of relativity.

Extracted knowledge graph:

Albert Einstein → born_in → Germany
Albert Einstein → developed → theory_of_relativity

Once two texts are converted into knowledge graphs, the library can compare them to measure how similar their factual content is.

This allows comparison between:

• context vs ground truth answers
• ground truth vs model-generated answers
• knowledge graphs extracted from different systems

---

Core Capabilities

Knowledge Graph Construction

Convert text paragraphs into structured knowledge graphs using LLM-based extraction.

Supported inputs include:

• single paragraphs
• multiple paragraphs
• CSV datasets containing text fields

---

Triplet-Level Graph Comparison

The library enables detailed comparison between two knowledge graphs by identifying:

• aligned triplets
• missing triplets
• extra triplets
• entity mismatches
• relation mismatches

This allows precise identification of where factual differences occur.

---

Semantic and Structural Graph Similarity

kganalytica measures similarity between two knowledge graphs using a hybrid approach that combines:

Semantic similarity
Triplet meaning is compared using sentence embeddings.

Structural similarity
Graph topology and relational structure are compared using graph kernels.

This allows similarity measurement beyond simple text matching.

---

Research Applications

The library is particularly useful for:

LLM Evaluation

Evaluate whether a language model's output preserves the factual relationships present in the context.

Knowledge Graph Research

Analyze and compare graph structures generated from different extraction models.

Information Extraction Systems

Benchmark triplet extraction pipelines and analyze their errors.

---

Installation

Install directly from PyPI:

pip install kganalytica

Setup API Keys

This library requires LLM API keys to construct knowledge graphs. Choose the provider(s) you want to use:

Local Development (with .env file)

Create a .env file in your project root:

GROQ_API_KEY=your_groq_api_key
OPENAI_API_KEY=your_openai_api_key
GEMINI_API_KEY=your_gemini_api_key

Google Colab / Cloud Environments

Since .env files aren't distributed with PyPI packages, set API keys directly in your notebook before importing the library:

Option 1: Set environment variables

import os
os.environ['GROQ_API_KEY'] = 'your_actual_groq_api_key_here'
os.environ['OPENAI_API_KEY'] = 'your_actual_openai_api_key_here'
os.environ['GEMINI_API_KEY'] = 'your_actual_gemini_api_key_here'

from kganalytica import construct_graph

Option 2: Create .env file in Colab

# Create .env file in Colab
with open('.env', 'w') as f:
    f.write('GROQ_API_KEY=your_api_key_here\n')
    f.write('OPENAI_API_KEY=your_api_key_here\n')
    f.write('GEMINI_API_KEY=your_api_key_here\n')

from kganalytica import construct_graph

Supported LLM Providers

• Groq - GROQ_API_KEY (supports llama-3.3-70b-versatile)
• OpenAI - OPENAI_API_KEY (supports GPT-4, GPT-3.5, etc.)
• Google Gemini - GEMINI_API_KEY (supports gemini-2.5-flash, etc.)
• HuggingFace - HF_API_KEY (supports Mistral, Llama, etc.)

---

Quick Example

from kganalytica import construct_graph

paragraph = '''
Albert Einstein was born in Germany.
He developed the theory of relativity.
'''

graph = construct_graph(paragraph)

for t in graph.triples:
    print(t.subject, "->", t.relation, "->", t.object)

Output:

Albert Einstein -> born_in -> Germany
Albert Einstein -> developed -> theory_of_relativity

---

Graph Similarity Example

from kganalytica import kg_similarity

kg1 = [
    ("Einstein", "born_in", "Germany"),
    ("Einstein", "developed", "Relativity")
]

kg2 = [
    ("Albert Einstein", "born in", "Germany"),
    ("Albert Einstein", "created", "Theory of Relativity")
]

score = kg_similarity(kg1, kg2)

print("Similarity:", score)

---

Project Motivation

The library was developed to support research on evaluating contextual understanding in large language models through knowledge graph comparison.

By analyzing information at the triplet level rather than the sentence level, researchers can better understand whether an AI system has captured the correct relationships between entities.

---

License

MIT License