Scan, redact, and manage PII in your documents before they get uploaded to a Retrieval Augmented Generation (RAG) system.
Project description
Open-source PII Detection for Retrieval Systems.
Scan, redact, and manage PII in your documents before they get uploaded to a Retrieval Augmented Generation (RAG) system.
Overview
DataFog works by scanning and redacting-out PII in files before are uploaded to a RAG system.
How it works
Installation
DataFog can be installed via pip:
pip install datafog # python client
Dev Notes
- Clone repo
- Run 'poetry install' to install dependencies (recommend entering poetry shell for preserving dependencies)
- Justfile commands:
just format
to apply formatting.just lint
to check formatting and style.just tag
to tag your project on gitjust upload
to publish to PyPi.
Testing
To run the datafog unit tests, check out this repository and do
tox
License
This software is published under the MIT license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
datafog-2.0.1.tar.gz
(4.6 kB
view hashes)