██████╗ ██████╗ ███████╗███╗   ██╗      ███╗   ██╗███████╗██╗    ██╗███████╗
██╔═══██╗██╔══██╗██╔════╝████╗  ██║      ████╗  ██║██╔════╝██║    ██║██╔════╝
██║   ██║██████╔╝█████╗  ██╔██╗ ██║█████╗██╔██╗ ██║█████╗  ██║ █╗ ██║███████╗
██║   ██║██╔═══╝ ██╔══╝  ██║╚██╗██║╚════╝██║╚██╗██║██╔══╝  ██║███╗██║╚════██║
╚██████╔╝██║     ███████╗██║ ╚████║      ██║ ╚████║███████╗╚███╔███╔╝███████║
 ╚═════╝ ╚═╝     ╚══════╝╚═╝  ╚═══╝      ╚═╝  ╚═══╝╚══════╝ ╚══╝╚══╝ ╚══════╝

🔁 Latest updates

[Unreleased] — v1.0.2 planned: live/streaming keyword search (search(refresh_interval=...)).

v1.0.1 — open-news/open-news-tui console commands (previously python -m only); CLI rewritten with full API coverage + fixed a search --sort popularity crash; TUI rewritten with a settings panel, richer search, save/open-in-browser; more robust source attribution — aggregator-hosted articles (MSN, Yahoo News, etc.) now attempt to resolve the original publisher instead of reporting the aggregator's domain; fixed full-content enrichment silently overwriting good search snippets with empty extracted fields.

v1.0.0 — Public API stabilized around fetch(), search(), get_article(), discover_and_get(), search_site(), batch_summarize(), summarize_text(). The old registry-backed live_news() / get_live_news() / search_news() / clear_feed_cache() interfaces from the 0.x series are retired — see docs/changelog.md for the full migration notes.

View the complete history in docs/changelog.md.

🎯 Features

📦 Installation

Note: the distribution name is open-news-api, not open-news — the latter was taken on PyPI.

Interactive installer (recommended)

curl -fsSL https://raw.githubusercontent.com/alphap365/open-news/main/install.sh | bash

Detects your OS (Linux/macOS/Termux/WSL), sets up an isolated environment, asks about the optional JS extra, and verifies the install before finishing. Full walkthrough: docs/installation.md.

With pip

pip install open-news-api
# a specific version:
pip install open-news-api==1.0.1

With uv

uv add open-news-api

# or drop it straight into a throwaway environment to try it:
uvx --from open-news-api open-news --help

From source (development)

git clone https://github.com/alphap365/open-news.git
cd open-news

# with pip
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
python -m pip install -e ".[dev]"

# or with uv (matches this repo's CI)
uv sync
uv run pytest -q

Optional: JavaScript rendering

For sites that render article content client-side:

pip install "open-news-api[js]"    # or: uv add "open-news-api[js]"
playwright install chromium         # one-time browser download, ~300MB

Then pass js=True to get_article(), discover_and_get(), fetch()/search() (with full_content=True), or batch_summarize(). If the extra isn't installed, it logs a warning and falls back to plain HTTP rather than raising.

Dependencies installed automatically: lxml • python-dateutil • httpx • beautifulsoup4 • feedparser • googlenewsdecoder • requests • ddgs • langdetect

🚀 Quick Start

1️⃣ Extract one article

from open_news import get_article

article = get_article("https://www.bbc.com/news/world-us-canada-12345678")
print(article["title"])
print(article["text"][:500])
print(f"Source: {article['source']}")
print(f"Published: {article['publish_date']}")

2️⃣ Search Google News

from open_news import search

results = search("artificial intelligence", query_mode="all", exclude_terms=["sports"], max_results=5)
for a in results:
    print(f"✓ {a['title']}")
    print(f"  → {a['url']}\n")

3️⃣ Live category/location news

from open_news import fetch

tech_news = fetch(category="tech", max_results=5)
india_news = fetch(location="in", max_results=5)   # location takes precedence over category

for a in tech_news:
    print(f"[{a['source']}] {a['title']}")

4️⃣ Discover any website

from open_news import discover_and_get

articles = discover_and_get("https://techcrunch.com", limit=5)
for a in articles:
    print(f"✓ {a['title']}")

RSS auto-discovery is tried first; the async crawler follows same-domain links (respecting robots.txt) if no feed is found.

5️⃣ Batch fetch & summarize

from open_news import batch_summarize

urls = ["https://example.com/article1", "https://example.com/article2"]
results = batch_summarize(urls, sentence_count=2, max_workers=3)

for r in results:
    if r["status"] == "success":
        print(f"📰 {r['title']}\n   {r['summary']}\n")
    else:
        print(f"❌ Failed: {r['error']}")

6️⃣ Search + summarize in one call

from open_news import search_and_summarize

results = search_and_summarize("climate change", limit=5, sentence_count=2)
for a in results:
    print(f"🔗 {a['url']}\n📰 {a['title']}\n   {a['summary']}\n")

7️⃣ Search a single domain

from open_news import search_site

results = search_site("budget", domain="reuters.com", limit=5)
for a in results:
    print(f"✓ {a['title']}\n  → {a['url']}\n")

8️⃣ Live polling

from open_news import fetch

# yields only newly-seen articles each poll; minimum interval is 5s
for new_articles in fetch(category="general", refresh_interval=60):
    for a in new_articles:
        print(a["title"])

9️⃣ Dedupe articles yourself

from open_news import dedupe_articles

raw = fetch(category="general", dedupe=False)
merged = dedupe_articles(raw, fuzzy=True)   # collapse same-story-different-outlet duplicates

🖥️ CLI & TUI

open-news fetch --category tech --limit 5
open-news search "AI regulation" --mode all --exclude sports
open-news discover https://www.bbc.com --limit 10
open-news summarize --query "climate policy" --sentences 2
open-news --version

Or launch the numbered menu — no flags to remember:

open-news-tui

• docs/cli-reference.md — every subcommand and flag
• docs/tui-guide.md — menu walkthrough, settings panel, live-refresh limitations

🔀 Function names: current vs legacy alias

Two names, same function, both stable — pick whichever reads better:

⚠️ Retired in v1.0.0, not aliases: live_news, get_live_news, search_news, clear_feed_cache. If you're upgrading from 0.x, see doc/changelog.md for the direct replacement of each.

📚 API Reference

Full parameter tables and return shapes for every function: docs/python-api.md

🧭 Known limitations

• Source attribution is best-effort on aggregator sites. MSN/Yahoo News/etc. don't consistently expose the original publisher in their markup; when no hint is found, the aggregator's own name is reported with meta.source_is_aggregator=True, meta.source_resolved=False so you can distinguish a confirmed publisher from a fallback.
• No live/streaming keyword search yet — refresh_interval exists on fetch() but not search(). Planned for v1.0.2.
• discover_and_get() has no whitelist/blacklist parameters yet, unlike fetch()/search().

Details and architecture context: docs/architecture.md.

🧪 Development

uv sync
uv run pytest -q
uv run python -m compileall -q open_news

# or with pip
python -m pip install -e ".[dev]"
pytest -q
python -m compileall -q open_news

CI runs this matrix across Python 3.10–3.13 on every push; see .github/workflows/ci.yml.

🤝 Contributing

Issues and PRs welcome:

• 🐛 Bug reports · ✨ Feature requests · 📝 Documentation · 💻 Pull requests

Please read CONTRIBUTING.md, keep changes focused, and add tests for behavior changes. Feed-discovery targets aren't curated centrally in this version — discover_and_get() works against any site directly.

🙏 Acknowledgements

Built on: httpx • lxml • feedparser • BeautifulSoup4 • googlenewsdecoder • ddgs • langdetect • Playwright (optional)

📄 License

MIT — see LICENSE.