OSINT GPT

osintgpt indexes documents an analyst already holds and answers questions from that material with citations. It is built for examining research collections without mixing projects or losing the path back to a source. Use it from the command line or as a Python library, with hosted or local models and SQLite, Qdrant, or Postgres storage. Embeddings support OpenAI, Gemini, Voyage, Ollama, and sentence-transformers; generation supports OpenAI, Gemini, Anthropic, and Ollama.

Install

The core install includes the document readers and command-line interface:

pip install osintgpt

Local embeddings are the one opt-in extra because they bring sentence-transformers and torch:

pip install osintgpt[local]

See osintgpt on PyPI for package metadata and release history.

Quickstart

By default, embeddings and answers use OpenAI. Set OPENAI_API_KEY and OPENAI_GPT_MODEL in your environment first, or configure the fully local path.

osintgpt project create "Research notes"
osintgpt project use research-notes
osintgpt add ./documents
osintgpt index
osintgpt ask "When is the launch date?"

A real run ends like this, with the temporary paths shortened:

1/1 …/documents/brief.md
1 documents, 1 chunks

The launch date is 14 March. [1]

Sources
• …/documents/brief.md

An unindexed project is also a valid state: ask explains that nothing was retrieved and does not call the generation model.

Keep data local

Fully local means the [local] extra for embeddings, Ollama running on your machine for generation, and the default SQLite store; nothing leaves your machine at query time. These are separate pieces: the Python extra installs sentence-transformers and torch, while Ollama is a separately installed local server and costs nothing from pip.

osintgpt config set embedding_provider sentence-transformers
osintgpt config set generation_provider ollama
osintgpt config set generation_model MODEL_NAME
osintgpt doctor

Replace MODEL_NAME with a model already available to Ollama. A model fetched on first use needs a network connection during setup; operation is local after that download. doctor runs offline by default and reports what would leave the machine, provider readiness, stored models, source coverage, and embedding model mismatches. Add --check-providers only when you want it to contact configured services.

What it reads

The 29 readable extensions cover PDF, Word, Excel and CSV, JSON and JSONL, Markdown and plain text, HTML and XML, and common image formats when the embedding model supports them. A fallback converter handles formats such as PowerPoint and EPUB. Structured files need a content-field mapping when they are registered; scanned PDF pages need a transcriber to recover text.

Use it as a library

The CLI quickstart is the shortest route. The same project and retrieval APIs are available to Python callers:

from osintgpt import Project, Settings, search_project
from osintgpt.llm import build_embedding_provider

project = Project.load("/path/to/project")
settings = project.settings_for(Settings.from_env(".env"))
embedder = build_embedding_provider(
    project.settings.embedding_provider, settings
)

for hit in search_project(project, "What does the evidence say?", embedder):
    print(hit.score, hit.chunk.citation)

Configuration is passed into library calls; importing osintgpt does not read the environment or select a project behind the caller's back.

Where projects live

The default home is ~/.osintgpt, with each project under projects/<slug>/. project.toml holds non-secret choices, sources.toml records registered locations and field mappings, and store.sqlite contains the default local vector store. Back up the project directory together with any registered material stored outside it; Qdrant and Postgres stores need their own backup.

Responsible use

osintgpt analyzes material supplied by its operator. Use it only with data you may lawfully process, protect personal and sensitive information, verify answers against their cited passages, and understand any costs or data handling terms of the providers you configure. The maintainers are not liable for misuse or third-party service charges.

License

Licensed under the Apache License 2.0.