Skip to main content

A local file-native graph layer for relationship-aware text retrieval.

Project description

FileGraphDB

FileGraphDB builds a local relationship graph over ordinary text files so an LLM can retrieve only the most relevant files instead of reading an entire folder.

Setup

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
pip install -e .

Build A File Graph

filegraph --folder . build

Files longer than 2,000 words are automatically split into overlapping chunk nodes:

big_report.txt#chunk-0001
big_report.txt#chunk-0002

You can tune this:

filegraph --folder ./docs build --chunk-threshold 2000 --chunk-words 800 --chunk-overlap 120

Or disable chunking:

filegraph --folder ./docs --chunk-threshold 0 build

Show strongest relationships:

filegraph --folder . edges

Find files related to one file:

filegraph --folder . related research/file_native_graph_database.md

Retrieve likely files for an LLM query:

filegraph --folder . search "How can file relationships reduce LLM token cost?"

Print LLM-ready context:

filegraph --folder . context "How can file relationships reduce LLM token cost?"

Python Library

from filegraphdb import FileGraphDB

graph = FileGraphDB("./docs")
graph.build()

for result in graph.retrieve("What caused the project delay?", limit=4):
    print(result.document.rel_path, result.score)

The first build is the expensive step. After that, the SQLite graph lives at:

.filegraphdb.sqlite

Optional Open-Source Embedding Model

By default, FileGraphDB uses local TF-IDF + LSA semantic vectors from scikit-learn. To use a stronger open-source embedding model:

pip install -e ".[models]"
filegraph --folder ./docs --use-model build

The default model is:

sentence-transformers/all-MiniLM-L6-v2

Small LLM Demo

The earlier local LLM demo is still available:

python small_llm.py

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

filegraphdb-0.1.0.tar.gz (14.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

filegraphdb-0.1.0-py3-none-any.whl (15.5 kB view details)

Uploaded Python 3

File details

Details for the file filegraphdb-0.1.0.tar.gz.

File metadata

  • Download URL: filegraphdb-0.1.0.tar.gz
  • Upload date:
  • Size: 14.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.9.12

File hashes

Hashes for filegraphdb-0.1.0.tar.gz
Algorithm Hash digest
SHA256 962c70484568e272a80f3daa9b7372c58b0cb62325aff7a81024fd233b136a5f
MD5 84737e23137e7c11c063b7e2cedba381
BLAKE2b-256 cd0c9e1d909e2c3a2919a7062b309ef48090da754fb23b7b9d62aa666dd9ac77

See more details on using hashes here.

File details

Details for the file filegraphdb-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: filegraphdb-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 15.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.9.12

File hashes

Hashes for filegraphdb-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f08fec4f36e48c1ca0ad7d11ab0360b84f0451123571a60ec71431f7c002989f
MD5 79c4bfc859f475be648289ed212f6d52
BLAKE2b-256 d797cf074fc2614d4ac7fe27f61c3043ba0f1d72c96066331232462f8d506dcf

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page