research-guild-ai 0.1.0

CLI for competitive intelligence research: collect, analyze, and export source-cited reports.

AI Researcher Crawler

Research any topic, company, or competitor from your terminal — and get cited, fact-checked reports you can export to Word, Excel, or PowerPoint.

Quick demo

$ pip install research-guild-ai
$ airc init
Which LLM provider would you like to use?
  1. OpenAI
  2. Google Gemini
  3. Anthropic Claude
Choice [1]: 2
Enter your Google Gemini API key:
Enter your Firecrawl API key:
Saved.

Setup complete. Config stored at ~/.config/ai-researcher-crawler/

$ airc collect
Writing to: ~/.local/share/ai-researcher-crawler/

=== AI Researcher Crawler — Scoping ===

What are you researching?

  1. Business / tech competitive intelligence (companies, products, markets)
  2. History (eras, past events, historical figures/topics)
  3. Current events / recent news
  4. General research (any other topic)

Choose a number [1]: 1
  → Mode: competitive

Research subject (be specific, e.g. 'HR SaaS' or 'AI code assistants' — not just 'market') []: Law Firm Management Systems
Target geography (comma-separated countries/regions, or leave blank for global): global
Tech keywords (comma-separated, e.g. llm, saas, mobile): saas, legaltech, practice-management
Known competitors (comma-separated company names; leave blank or type 'none' to discover): Clio, Amberlo, LEAP, NetDocuments
Research objective (what decision or question should this answer?) []: Compare offerings, pricing signals, and geographic focus
Research depth (quick | deep) [quick]: quick
Output formats (comma-separated, e.g. markdown_report, word file, xlsx): markdown_report, docx

Scope saved: ~/.local/share/ai-researcher-crawler/scope/scope_20260714T095055Z.json

Starting collection (competitor pages + web + GitHub)...

=== Collection Summary ===
Scope saved to: ~/.local/share/ai-researcher-crawler/scope/scope_20260714T095055Z.json
Research mode: competitive
Competitors: Clio, Amberlo, LEAP, NetDocuments
Raw data directory: ~/.local/share/ai-researcher-crawler/raw/20260714T095311Z

Collected:
  - Clio (homepage): https://www.clio.com/ → .../clio-homepage.md
  - Amberlo (homepage): https://www.amberlo.io/ → .../amberlo-homepage.md
  - LEAP (homepage): https://www.leaplegalsoftware.com/ → .../leap-homepage.md
  - NetDocuments (homepage): https://www.netdocuments.com/ → .../netdocuments-homepage.md
  - web (web-search): .../web-search.md
  - github (github-search): .../github-search.md

$ airc analyze \
    --scope-file ~/.local/share/ai-researcher-crawler/scope/scope_20260714T095055Z.json \
    --raw-dir ~/.local/share/ai-researcher-crawler/raw/20260714T095311Z

=== Analysis Summary ===
Documents analyzed: 12
Confidence: medium
Research loops used: 0
Report saved to: ~/.local/share/ai-researcher-crawler/reports/report_20260714T095311Z.md

# From the report (excerpt):
# Generative and agentic AI are becoming core platform features
# (e.g. Clio Work, ndMAX) [clio/features, https://...; netdocuments/features, https://...]
#
# | AI Capabilities | Clio Work ... | *Not found in collected sources* | ...

Install

Requires Python 3.11+.

pip install research-guild-ai

Until the package is on PyPI, install from GitHub or a local wheel:

# From this repo (subdirectory install)
pip install "git+https://github.com/Manishkarki1/ai-researcher.git#subdirectory=ai-researcher-crawler"

# Or build a wheel locally, then:
#   cd ai-researcher-crawler && python -m build
pip install dist/ai_researcher_crawler-*.whl

Confirm the CLI is on your PATH:

airc --help

Setup

airc init

This asks which LLM provider to use (OpenAI, Google Gemini, or Anthropic Claude), prompts for that provider’s API key, then asks for a Firecrawl API key (used to collect public web sources). Keys are stored in your OS credential store when available — not in a plaintext project .env. If the OS keyring is unavailable, they fall back to a mode-0600 file under the config directory.

Get keys here:

Service	Signup
Firecrawl	https://www.firecrawl.dev/
OpenAI	https://platform.openai.com/api-keys
Google Gemini	https://aistudio.google.com/apikey
Anthropic	https://console.anthropic.com/

You can also set keys later with airc config set-key, or via environment variables for CI (see Configuration).

Research modes

Mode	Use it for
Competitive	Comparing companies, products, markets
History	Past events, eras, historical figures
Current events	Recent news, ongoing situations
General	Anything else

Basic usage

Collect sources

airc collect

Interactive prompts pick a mode, save a scope file, optionally discover competitors (competitive mode), and scrape public pages into a timestamped raw folder.

=== Collection Summary ===
Scope saved to: ~/.local/share/ai-researcher-crawler/scope/scope_20260714T095055Z.json
Research mode: competitive
Competitors: Clio, Amberlo, LEAP, NetDocuments
Raw data directory: ~/.local/share/ai-researcher-crawler/raw/20260714T095311Z

Collect and analyze in one step:

airc collect --analyze

Analyze into a report

airc analyze \
  --scope-file ~/.local/share/ai-researcher-crawler/scope/scope_20260714T095055Z.json \
  --raw-dir ~/.local/share/ai-researcher-crawler/raw/20260714T095311Z

=== Analysis Summary ===
Documents analyzed: 12
Confidence: medium
Research loops used: 0
Report saved to: ~/.local/share/ai-researcher-crawler/reports/report_20260714T095311Z.md

Export

airc export \
  --report ~/.local/share/ai-researcher-crawler/reports/report_20260714T095311Z.md \
  --formats docx,xlsx,pptx

=== Exports ===
  - ~/.local/share/ai-researcher-crawler/exports/report_20260714T095311Z.docx
  - ~/.local/share/ai-researcher-crawler/exports/report_20260714T095311Z.xlsx
  - ~/.local/share/ai-researcher-crawler/exports/report_20260714T095311Z.pptx

What you get

Every claim is cited and confidence-scored. When something isn’t in the collected sources, the report says so (not found in collected sources / not publicly disclosed) instead of guessing. Single-source claims are flagged as unverified.

Exports: Markdown, Word (docx), Excel (xlsx), and PowerPoint (pptx).

Entity deep-dive

After analysis, the tool lists people and organizations mentioned in the research and lets you follow up on selected ones. Public figures and organization representatives can get a fuller follow-up research pass. Private individuals mentioned in coverage get a limited, source-only summary — the tool will not go digging for more about a private person, and those names are not written into the cross-run entities index.

Detected entities:
  1. Clio — legal practice management vendor (organization_representative) — full deep-dive available
  2. Jane Doe — mentioned in news coverage (incident_victim_or_private_citizen) — known-information summary only, no new research

Select entities (comma-separated numbers), or:
  all-public   → select all public_figure / organization_representative entities
  skip         → skip
Selection [skip]:

Monitoring / re-running research

Re-run analysis with --compare-to <prior_run_id> to insert a What Changed section against a previous run’s snapshot:

airc analyze \
  --scope-file ~/.local/share/ai-researcher-crawler/scope/scope_YYYYMMDDTHHMMSSZ.json \
  --raw-dir ~/.local/share/ai-researcher-crawler/raw/YYYYMMDDTHHMMSSZ \
  --compare-to 20260712T173833Z

What Changed section inserted (vs 20260712T173833Z)

Configuration reference

airc config show          # provider, model, masked key status, paths
airc config set-key openai|gemini|anthropic|firecrawl
airc config reset         # clear stored config + credentials

Data directory (scopes, raw scrapes, reports, exports):

Priority	Source
1	airc --data-dir PATH …
2	AIRC_DATA_DIR
3	OS user data dir (e.g. ~/.local/share/ai-researcher-crawler/)

CI / non-interactive fallbacks (no prompts):

export FIRECRAWL_API_KEY=...
export LLM_PROVIDER=openai   # or gemini | anthropic
export OPENAI_API_KEY=...    # or GOOGLE_API_KEY / ANTHROPIC_API_KEY
export AIRC_DATA_DIR=/var/lib/airc

CLI flags (--firecrawl-api-key, --llm-provider, --openai-api-key, …) override env for a single run.

Limitations

• Collection respects site access norms (including robots.txt / terms of service via the crawl provider), so some sites will not be scraped.
• LLM analysis can still make mistakes; the fact-checking design reduces silent invention, it does not eliminate all error.
• deep research depth costs more in Firecrawl and LLM usage than quick.
• Competitor discovery is heuristic — always review the proposed list before accepting it.
• The package is early (0.1.x); expect CLI and output shape to evolve.

Ethics / responsible use

This tool is meant for public information only. It does not bypass logins, paywalls, or CAPTCHAs. It includes safeguards so private individuals mentioned in coverage are not turned into dug-up personal profiles from aggregated public data. Use it for organizational research, public figures, and topics — not for building dossiers on private people.

For the full anti-hallucination design (citations, claim kinds, reviewer loop), see DOCUMENTATION.md §11.

Contributing

Issues and PRs welcome. Architecture, pipeline internals, and module deep-dives live in DOCUMENTATION.md — start there if you want to change how the system works.

License

MIT