twag 0.1.38

Twitter aggregator for market-relevant signals

twag 📊

Twitter/X market-signal aggregation with LLM-powered triage, enrichment, and digest generation.

twag pulls tweets from your timeline, scores them for market relevance using LLMs, enriches high-signal content, and generates daily digests. It's designed for traders and investors who want signal without the noise.

Example telegram notification. Chart emojis are links to tweets that have charts:

Features

• Smart Scoring — LLM-powered triage (0-10) with category tagging
• Full-Text Search — SQLite FTS5 with boolean queries and filters
• X Article Summaries — Extracts key points and action items from long-form posts
• Telegram Alerts — Real-time notifications for high-signal tweets
• Web Feed — FastAPI + React UI for browsing processed tweets
• OpenClaw Skill — Native integration with OpenClaw agents

Prerequisites

bird CLI (Required)

twag uses bird to access Twitter/X. Install it first:

# npm (recommended)
npm install -g @steipete/bird

# or pnpm/bun
pnpm add -g @steipete/bird
bun add -g @steipete/bird

# or Homebrew (macOS)
brew install steipete/tap/bird

Verify installation:

bird --version

Twitter Authentication

bird uses cookie-based auth. You need two cookies from your browser:

1. AUTH_TOKEN — Your auth_token cookie
2. CT0 — Your ct0 cookie

Getting Your Cookies

1. Log into x.com in your browser
2. Open DevTools (F12) → Application → Cookies → https://x.com
3. Copy the values of auth_token and ct0

Set them as environment variables:

export AUTH_TOKEN="your_auth_token_here"
export CT0="your_ct0_here"

Or add to ~/.env (twag will source this automatically):

echo 'export AUTH_TOKEN="..."' >> ~/.env
echo 'export CT0="..."' >> ~/.env

Verify auth works:

bird whoami

LLM API Keys

Required:

• GEMINI_API_KEY — Google AI Studio (free tier available)

Optional:

• ANTHROPIC_API_KEY — For higher-quality enrichment on high-signal tweets
• DEEPSEEK_API_KEY — For DeepSeek text triage/enrichment models

export GEMINI_API_KEY="your_gemini_key"
export ANTHROPIC_API_KEY="your_anthropic_key"  # optional
export DEEPSEEK_API_KEY="your_deepseek_key"    # optional

Installation

From PyPI (Recommended)

pip install twag

From Source

git clone https://github.com/clifton/twag.git
cd twag
pip install -e .

Using uv

uv pip install twag
# or from source
uv pip install -e .

Quick Start

# 1. Initialize config and database
twag init

# 2. Verify dependencies and environment
twag doctor

# 3. Fetch tweets from your timeline
twag fetch

# 4. Score and process tweets
twag process

# 5. Generate today's digest
twag digest --stdout

# 6. Search for specific topics
twag search "fed rate" --today

Core Workflow

FETCH → PROCESS → DIGEST

1. Fetch — Pull tweets from home timeline, tier-1 accounts, bookmarks
2. Process — Score tweets with LLM, categorize, enrich high-signal content
3. Digest — Generate markdown summaries grouped by theme

CLI Reference

Setup Commands

twag init              # Initialize config and database
twag init --force      # Overwrite existing config file
twag doctor            # Check dependencies and environment

Fetch Commands

# Default: home timeline + tier-1 accounts + bookmarks
twag fetch

# Single tweet (ID or URL)
twag fetch 1234567890123456789
twag fetch https://x.com/user/status/1234567890123456789

# User timeline
twag fetch --source user --handle @NickTimiraos --count 50

# Search
twag fetch --source search --query "Fed Powell rate" --count 30

# Control sources
twag fetch --no-tier1 --no-bookmarks

# Tier-1 rotation and pacing
twag fetch --stagger 5          # Only fetch 5 least-recent tier-1 accounts
twag fetch --delay 5.0          # 5s delay between tier-1 fetches (default: 3s)

Process Commands

twag process                    # Process unscored tweets (no alerts by default)
twag process --limit 100        # Limit batch size
twag process --dry-run          # Preview only
twag process --notify           # Send Telegram alerts

# Quote/reply reprocessing
twag process --no-reprocess-quotes       # Skip reprocessing dependency tweets
twag process --reprocess-min-score 5     # Min score for reprocessing (default: 3)

# Process specific tweet
twag process 1234567890123456789

Search Commands

# Basic search
twag search "inflation fed"

# With filters
twag search "rate hike" --category fed_policy
twag search "NVDA" --author zerohedge
twag search "earnings" --ticker AAPL

# Time filters
twag search "breaking" --today              # Since last market close
twag search "fed" --time 7d                 # Last 7 days
twag search "macro" --since 2026-01-15      # From specific date

# Output formats
twag search "fed" --format brief            # Compact output
twag search "fed" --format full             # Digest-style
twag search "fed" --format json             # JSON output

# Score threshold
twag search "market" --min-score 7          # High-signal only

# Additional filters
twag search --bookmarks                     # Only bookmarked tweets
twag search "fed" --tier 1                  # Filter by signal tier
twag search "fed" --order score             # Sort by: rank, score, or time

Query syntax:

• Simple: inflation fed (matches both)
• Phrase: "rate hike" (exact match)
• Boolean: inflation AND fed, fed NOT fomc
• Prefix: infla* (wildcard)

Narrative Commands

twag narratives list            # List active narratives with mention counts

Analyze Commands

twag analyze 1234567890123456789              # Analyze by ID
twag analyze https://x.com/user/status/123    # Analyze by URL
twag analyze 1234567890123456789 --reprocess  # Force re-analyze
twag analyze 1234567890123456789 -m gemini-2.0-flash  # Override model
twag analyze 1234567890123456789 --thread              # Persist the full conversation thread
twag analyze 1234567890123456789 --replies --reply-depth 2  # Persist target replies + one nested level
twag analyze 1234567890123456789 --thread --replies \
  --reply-depth 2 --max-reply-nodes 25 --max-pages 5

By default, twag analyze retains its original target-only behavior. --thread adds the conversation returned by bird thread; --replies performs a breadth-first reply traversal seeded by the target, or by every fetched thread status when combined with --thread. --reply-depth 1 means direct replies only, while 2 adds one nested level. --max-reply-nodes is a global cap on both reply statuses stored and reply-source requests visited. --max-pages caps every Bird thread/replies request; without it, explicitly requested context uses all available pages.

The target, thread statuses, and replies use the same storage path, so reply relationships, conversation IDs, links, media, and X Article fields are persisted consistently. Only the requested target is classified/reprocessed and printed; context is stored for downstream extraction and research. An explicit context-fetch failure exits nonzero instead of silently continuing with target-only data.

Digest Commands

twag digest                     # Generate today's digest
twag digest --date 2026-02-06   # Specific date
twag digest --stdout            # Output to terminal
twag digest --min-score 6       # Custom threshold

Account Management

twag accounts list              # All tracked accounts
twag accounts list --tier 1     # Tier-1 only
twag accounts add @handle       # Add account
twag accounts add @handle -t 1  # Add as tier-1
twag accounts promote @handle   # Promote to tier-1
twag accounts demote @handle    # Demote to tier-2
twag accounts mute @handle      # Mute account
twag accounts boost @handle --amount 10  # Boost weight
twag accounts decay             # Apply daily decay
twag accounts import            # Import from following.txt

Stats & Maintenance

twag stats                      # All-time stats
twag stats --today              # Today's stats

twag inference usage            # LLM token/cost report for last 30 days
twag inference usage --all-time # Include all logged inference usage

twag prune --days 14            # Delete old tweets
twag prune --dry-run            # Preview prune

twag export --days 7            # Export recent data

Narratives

twag narratives list            # List active narratives

Database Commands

twag db path                    # Show database location
twag db shell                   # Open SQLite shell
twag db init                    # Initialize/reset database
twag db rebuild-fts             # Rebuild search index
twag db dump                    # Backup database (auto-named file)
twag db dump --stdout           # Backup to stdout
twag db restore backup.sql      # Restore from backup
twag db restore backup.sql --force  # Restore without confirmation

Web Interface

twag web                        # Start web UI (localhost:5173)
twag web --host 127.0.0.1       # Bind to localhost only
twag web --port 8080            # Custom port
twag web --dev                  # Dev mode (Vite + hot reload)
twag web --no-reload            # Disable auto-reload on code changes

⚠️ Security: The web interface has no authentication. Only run on trusted networks or bind to localhost.

Configuration

twag config show                # Show current config
twag config path                # Show config file path
twag config set llm.triage_model gemini-2.0-flash
twag config set scoring.alert_threshold 8
twag config set scoring.min_score_for_analysis 6
twag config set scoring.max_article_summary_chars 20000

Data Paths

twag follows XDG defaults:

Path	Purpose
~/.config/twag/config.json	Configuration
~/.local/share/twag/twag.db	SQLite database
~/.local/share/twag/digests/	Generated digests
~/.local/share/twag/following.txt	Followed accounts

Override with TWAG_DATA_DIR environment variable.

Scoring System

Tweets are scored 0-10:

Score	Signal Level	Behavior
8-10	Alert	Telegram alert (if configured)
7	High signal	Enriched, included in digests
5-6	Market relevant	Included in digests
3-4	News/context	Searchable, not in digests
0-2	Noise	Stored but filtered out

Categories

fed_policy, inflation, job_market, macro_data, earnings, equities, rates_fx, credit, banks, consumer_spending, capex, commodities, energy, metals_mining, geopolitical, sanctions, tech_business, ai_advancement, crypto, noise

Automation

Data Collection (systemd timer)

Create ~/.config/systemd/user/twag-aggregator.service:

[Unit]
Description=TWAG Twitter Aggregator
After=network.target

[Service]
Type=oneshot
ExecStart=%h/.local/bin/twag fetch && %h/.local/bin/twag process
WorkingDirectory=%h
EnvironmentFile=%h/.env

Create ~/.config/systemd/user/twag-aggregator.timer:

[Unit]
Description=Run TWAG every 15 minutes

[Timer]
OnBootSec=5min
OnUnitActiveSec=15min
Persistent=true

[Install]
WantedBy=timers.target

Enable:

systemctl --user daemon-reload
systemctl --user enable --now twag-aggregator.timer

macOS (launchd)

See SUGGESTED_CRON_SCHEDULE.md for launchd plist examples.

Telegram Alerts

1. Create a bot via @BotFather
2. Get your chat ID by messaging the bot
3. Set environment variables:

export TELEGRAM_BOT_TOKEN="your_bot_token"
export TELEGRAM_CHAT_ID="your_chat_id"

Tweets scoring 8+ will trigger alerts when you run twag process --notify.

OpenClaw Integration

twag is an OpenClaw skill. Once installed, agents can:

# Search for market-relevant tweets
twag search "fed rate" --today -s 7

# Generate digests
twag digest --stdout

# Analyze specific tweets
twag analyze https://x.com/user/status/123

Skill Installation

The skill is auto-discovered if twag is in PATH. For manual setup:

# Link skill to OpenClaw
ln -s /path/to/twag ~/.openclaw/skills/twag

Scheduled Digests

See SUGGESTED_CRON_SCHEDULE.md for OpenClaw cron job examples that deliver digests to Telegram.

Environment Variables

Variable	Required	Purpose
AUTH_TOKEN	Yes	Twitter auth cookie
CT0	Yes	Twitter ct0 cookie
GEMINI_API_KEY	Yes	LLM triage/vision
ANTHROPIC_API_KEY	No	Enhanced enrichment
DEEPSEEK_API_KEY	No	DeepSeek text triage/enrichment
TELEGRAM_BOT_TOKEN	No	Alert delivery
TELEGRAM_CHAT_ID	No	Alert destination
TWAG_DATA_DIR	No	Override data directory

Troubleshooting

"bird not found"

# Check if bird is installed
which bird

# Install if missing
npm install -g @steipete/bird

"Authentication failed" / 401 errors

# Verify cookies are set
echo $AUTH_TOKEN
echo $CT0

# Test bird auth
bird whoami

# If expired, get fresh cookies from browser

"Query IDs stale" / 404 errors

# Refresh GraphQL query cache
bird query-ids --fresh

"GEMINI_API_KEY not set"

# Verify key is exported
echo $GEMINI_API_KEY

# Test with curl
curl "https://generativelanguage.googleapis.com/v1/models?key=$GEMINI_API_KEY"

Database issues

# Check database location
twag db path

# Rebuild search index
twag db rebuild-fts

# Backup and restore
twag db dump backup.sql
twag db restore backup.sql --force

Development

# Install dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Lint and format
ruff check .
ruff format .

# Frontend development
cd twag/web/frontend
npm install
npm run dev

License

MIT. See LICENSE.

Links

• Repository: https://github.com/clifton/twag
• Issues: https://github.com/clifton/twag/issues
• bird CLI: https://github.com/steipete/bird
• OpenClaw: https://github.com/openclaw/openclaw