stockholm-finance 1.1.15

Advanced financial sentiment analysis tool with policy impact assessment

📊 Stockholm Finance

Advanced financial sentiment analysis with interactive terminal dashboard, intelligent caching, and real-time market data integration.

🚀 Quick Start

# Install
pip install stockholm-finance

# Run
stockholm-finance

✨ Features

• 🎯 Sentiment Analysis: NLP-powered market sentiment from financial news
• 📊 Interactive Dashboard: Professional terminal UI with real-time data
• 💾 Smart Caching: TTL-based caching for optimal performance
• 🏛️ Policy Analysis: Government policy impact assessment
• 📈 Live Charts: Interactive price history and trend visualization
• 🔄 Real-time Updates: Live market data integration

📖 Usage

Command Line

stockholm-finance                    # Launch interactive dashboard
stockholm-finance --quick            # Quick startup mode (fewer tickers)
stockholm-finance --verbose          # Debug information
stockholm-finance --warm-cache       # Pre-warm cache for faster startup
stockholm-finance --quick --warm-cache  # Quick mode with cache warming

Performance Optimization

Stockholm includes several performance optimizations for faster startup:

• 🔥 Hot Loading: Dashboard appears immediately while data loads progressively in background
• 🚀 Proactive Cache System: Revolutionary cache that refreshes data BEFORE it expires (zero user delays)
• 📦 Intelligent Caching: Reduces API calls by 85%+ with smart TTL-based caching and priority refresh
• ⚡ Background Data Fetching: Continuous cache warming and proactive refresh while dashboard is running
• 🎯 Priority-Based Refresh: Frequently accessed data gets refreshed more aggressively
• 🔄 Cache Pre-warming: Use --warm-cache to populate cache before startup

# For fastest startup experience with proactive cache:
stockholm-finance --quick --warm-cache

# Test the proactive cache system:
python test_proactive_detailed.py

# Benchmark performance improvements:
python scripts/benchmark_startup.py

Python API

from src.core.financial_analyzer import main
main()  # Launch dashboard programmatically

🏗️ Development

Setup

git clone https://github.com/aykaym/stockholm.git
cd stockholm
pip install -e ".[dev]"

Testing

pytest tests/ -v                    # Run tests
pytest tests/ --cov=src            # With coverage

Code Quality

trunk check --all                  # Lint all files
trunk fmt                          # Format code

🚀 Releases

Releases are automated via GitHub Actions:

git tag v1.0.x                     # Create version tag
git push origin v1.0.x             # Trigger release pipeline

The pipeline automatically:

• ✅ Runs tests and quality checks
• ✅ Builds clean distribution packages
• ✅ Publishes to PyPI as stockholm-finance
• ✅ Creates GitHub release with artifacts
• ✅ Builds and pushes Docker image

🤖 AI Assistant Context

For AI assistants working with this codebase:

Code Style & Quality

• Linter: Uses Trunk for code quality (Black + Ruff)
• Pre-commit hooks: Automatically run trunk check --fix before commits
• Style: Follow Black formatting, 88-character line limit
• Imports: Use isort for import organization

Project Structure

src/
├── core/                     # Financial analysis engines
│   ├── financial_analyzer.py # Main entry point & orchestration
│   ├── sentiment_analyzer.py # NLP-based market sentiment analysis
│   ├── policy_analyzer.py    # Government policy impact analysis
│   └── earnings_fetcher.py   # Quarterly earnings data fetching
├── data/                     # Data fetching and caching
│   ├── data_fetcher.py       # Multi-source data aggregation
│   ├── cached_data_fetcher.py# Cached API wrappers
│   └── cache_manager.py      # Smart TTL cache management
├── ui/                       # Textual dashboard interface
│   ├── textual_dashboard.py  # Interactive TUI dashboard
│   └── display_utils.py      # Terminal output utilities
└── config/                   # Configuration management
    └── config.py             # Application configuration

Key Technologies

• UI Framework: Textual for terminal interface
• Charts: textual_plotext for interactive terminal plots
• Data: yfinance for market data, feedparser for news
• NLP: textblob for sentiment analysis
• Caching: Custom TTL-based file caching system

Core Systems

Performance Optimization System

Stockholm includes advanced performance optimizations for instant startup and responsive user experience.

🔥 Hot Loading Architecture:

# Progressive data loading phases
LOADING_PHASES = {
    'critical': ['prices', 'basic_market_data'],     # Load first (2-3s)
    'secondary': ['news', 'government_data'],        # Load second (5-8s)
    'analysis': ['sentiment', 'policy_analysis'],    # Complete analysis (10-15s)
}

# Background data fetching
class BackgroundDataFetcher:
    def start_background_fetching(self):
        # Continuous cache warming while dashboard runs
        # Updates UI components as fresh data arrives
        # Reduces perceived startup time by 70%

⚡ Startup Performance Comparison:

• Cold Start (empty cache): ~25-30 seconds
• Warm Start (pre-warmed cache): ~8-12 seconds
• Hot Loading (progressive): ~2-3 seconds to UI, data loads in background

🚀 Performance Features:

1. Immediate UI Display: Dashboard appears in 2-3 seconds
2. Progressive Data Loading: Critical data first, analysis second
3. Background Cache Warming: Continuous data ingestion while running
4. Smart Cache Pre-warming: --warm-cache flag for fastest startup

Intelligent Caching System

The caching system is the performance backbone of Stockholm, achieving 85% API call reduction.

Architecture:

# TTL Configuration (in minutes)
CACHE_TTL = {
    'news': 15,           # Financial news updates
    'prices': 5,          # Real-time stock prices
    'company_names': 1440, # Company metadata (24 hours)
    'market_data': 10,    # Market indices
    'policy_news': 30,    # Government announcements
    'analyst_data': 60,   # Analyst recommendations
    'earnings': 1440,     # Quarterly earnings data
}

Cache Key Generation:

• Uses MD5 hashes of request parameters
• Format: {data_type}_{hash}.cache
• Example: prices_a1b2c3d4.cache for AAPL price data

Performance Metrics:

• Cold cache: ~81 API calls for full dashboard
• Warm cache: ~40 API calls (51% reduction)
• Hot cache: ~25 API calls (69% reduction)
• Cache hit rate: 70%+ on subsequent runs

Cache Management:

# Monitor cache performance
python tools/cache_monitor.py --stats

# Clear specific cache types
python tools/cache_monitor.py --clear news

# Emergency cache clear
python tools/clear_all_cache.py

Implementation Details:

• File-based storage in cache/ directory
• Atomic writes to prevent corruption
• Request tracking in request_log.json
• Automatic cleanup of expired entries
• Thread-safe operations for concurrent access

Sentiment Analysis Engine

Advanced NLP system for market sentiment analysis with multi-ticker support.

Core Algorithm:

def analyze_sentiment_around_ticker(text, ticker):
    """Context-aware sentiment analysis around ticker mentions."""
    # 1. Find ticker positions in text
    # 2. Extract surrounding context (±50 words)
    # 3. Apply TextBlob sentiment analysis
    # 4. Weight by proximity to ticker mention
    # 5. Return sentiment score (-1 to +1)

Multi-Ticker Detection:

• Regex patterns for ticker identification: r'\b[A-Z]{1,5}\b'
• Context validation against known ticker list
• Handles edge cases: $AAPL, AAPL:, (AAPL)
• Filters false positives (common words like "IT", "US")

Batch Processing:

• Processes 100+ articles in parallel
• Memory-efficient streaming for large datasets
• Error handling for malformed articles
• Progress tracking for long operations

Sector Classification:

SECTOR_MAPPING = {
    'AAPL': 'Technology',
    'MSFT': 'Technology',
    'JPM': 'Financial Services',
    'JNJ': 'Healthcare',
    # ... 130+ tickers mapped
}

Conflict Detection:

• Identifies articles mentioning competing stocks
• Calculates sentiment divergence between tickers
• Flags potential market-moving conflicts
• Used for risk assessment and correlation analysis

Policy Analysis System

Government policy impact assessment with weighted sentiment analysis.

Data Sources:

• Federal Reserve RSS feeds (5 different categories)
• Treasury Department announcements
• SEC regulatory updates
• Congressional financial committee news

Impact Classification Algorithm:

POLICY_KEYWORDS = {
    'high_impact': [
        'interest rate', 'federal funds rate', 'monetary policy',
        'quantitative easing', 'inflation target', 'recession',
        'fomc', 'rate hike', 'rate cut', 'dovish', 'hawkish'
    ],
    'medium_impact': [
        'banking regulation', 'stress test', 'capital requirements',
        'liquidity', 'financial stability', 'systemic risk'
    ],
    'sector_specific': [
        'energy policy', 'healthcare reform', 'tax policy',
        'trade policy', 'infrastructure', 'climate policy'
    ]
}

def calculate_policy_impact(article):
    """Calculate policy impact score (0-5 scale)."""
    # 1. Keyword matching with weights
    # 2. Source credibility scoring
    # 3. Recency factor (newer = higher impact)
    # 4. Article length consideration
    # 5. Return weighted impact score

Combined Analysis Formula:

final_sentiment = (market_sentiment * 0.7) + (policy_sentiment * 0.3)

Policy Categories:

• Monetary Policy: Fed decisions, interest rates, QE
• Regulatory: Banking rules, compliance, oversight
• Enforcement: Penalties, investigations, sanctions
• Economic: GDP, employment, inflation data

Interactive Dashboard

Professional terminal UI built with Textual framework.

Tab Architecture:

1. Overview Tab: Market summary, top performers, sector analysis
2. Tickers Tab: Interactive table with 130+ stocks, real-time data
3. News Tab: Categorized news tree with sentiment visualization
4. Policy Tab: Government policy analysis and impact assessment
5. Indices Tab: Market indices and sector performance

Real-time Update System:

class DashboardApp(App):
    def __init__(self):
        self.refresh_timer = self.set_interval(300, self.refresh_data)  # 5 min

    async def refresh_data(self):
        """Background data refresh without blocking UI."""
        # 1. Check cache validity
        # 2. Fetch only expired data
        # 3. Update UI components
        # 4. Show progress indicators

Interactive Elements:

• Sortable Tables: Click column headers to sort by any metric (ascending/descending)
• Adjustable Columns: Click and drag column borders to resize columns in real-time
• Modal Dialogs: Detailed ticker information with charts
• Progress Bars: Real-time feedback during data operations
• Status Bar: Connection status, last update time, cache stats

Column Sorting Features:

• Click any column header to sort by that column
• First click: Sort ascending (↑ arrow indicator)
• Second click: Sort descending (↓ arrow indicator)
• Visual indicators: Arrows show current sort column and direction
• Smart sorting: Handles different data types (numbers, text, dates)
• Available in all tables: Tickers, News, Headlines, Policy articles

Sortable Columns:

• Tickers Table: Rank, Ticker, Price, Change, Sentiment, Articles, Sector
• News Tables: #, Headline, Time, Sentiment, Score, Tickers, Source
• Real-time updates: Sorting persists during data refreshes

Keyboard Navigation:

BINDINGS = [
    ("q", "quit", "Quit"),
    ("r", "refresh", "Refresh Data"),
    ("1", "show_tab('overview')", "Overview Tab"),
    ("2", "show_tab('tickers')", "Tickers Tab"),
    ("3", "show_tab('news')", "News Tab"),
    ("4", "show_tab('policy')", "Policy Tab"),
    ("ctrl+e", "export_data", "Export Data"),
    ("ctrl+r", "reset_column_widths", "Reset Column Widths"),
]

Color Coding System:

• Green: Positive sentiment (>0.1), price increases (>1%)
• Red: Negative sentiment (<-0.1), price decreases (<-1%)
• Yellow: Neutral sentiment (-0.1 to 0.1), minimal price change
• Blue: Policy-related content, high-impact news
• Gray: No data, loading states, disabled elements

Performance Optimizations:

• Lazy loading for large datasets
• Virtual scrolling for ticker tables
• Debounced user input handling
• Efficient re-rendering with dirty checking
• Background data fetching to prevent UI blocking

Data Flow Architecture

Request Processing Pipeline

graph TD
    A[User Request] --> B[Cache Check]
    B -->|Hit| C[Return Cached Data]
    B -->|Miss| D[API Request Queue]
    D --> E[Rate Limiting]
    E --> F[External API Call]
    F --> G[Data Validation]
    G --> H[Cache Storage]
    H --> I[Return Fresh Data]
    C --> J[UI Update]
    I --> J

API Integration Layer

Primary Data Sources:

DATA_SOURCES = {
    'yfinance': {
        'endpoints': ['info', 'history', 'news', 'recommendations'],
        'rate_limit': '2000/hour',
        'reliability': '95%',
        'cache_strategy': 'aggressive'
    },
    'feedparser': {
        'feeds': ['fed_press', 'fed_monetary', 'fed_speeches', 'fed_banking'],
        'rate_limit': 'unlimited',
        'reliability': '99%',
        'cache_strategy': 'moderate'
    },
    'textblob': {
        'type': 'local_processing',
        'rate_limit': 'unlimited',
        'cache_strategy': 'none'
    }
}

Error Handling Strategy:

def robust_api_call(func, *args, **kwargs):
    """Robust API calling with exponential backoff."""
    for attempt in range(3):
        try:
            return func(*args, **kwargs)
        except (ConnectionError, Timeout) as e:
            wait_time = 2 ** attempt  # Exponential backoff
            time.sleep(wait_time)
        except RateLimitError:
            time.sleep(60)  # Wait 1 minute for rate limit reset
        except Exception as e:
            log_error(f"API call failed: {e}")
            return None
    return None  # All retries failed

Data Validation Pipeline:

1. Schema Validation: Ensure required fields exist
2. Type Checking: Validate data types (float for prices, etc.)
3. Range Validation: Check for reasonable values (prices > 0)
4. Completeness Check: Verify minimum data requirements
5. Sanitization: Clean text data, remove HTML tags

Concurrent Processing System

async def fetch_multiple_tickers(tickers):
    """Concurrent data fetching with controlled parallelism."""
    semaphore = asyncio.Semaphore(10)  # Max 10 concurrent requests

    async def fetch_single(ticker):
        async with semaphore:
            return await cached_data_fetcher.get_ticker_data(ticker)

    tasks = [fetch_single(ticker) for ticker in tickers]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    return process_results(results)

Memory Management:

• Streaming processing for large datasets
• Garbage collection after batch operations
• Memory-mapped files for large cache entries
• Lazy loading of non-critical data

Development Workflow

CI/CD Pipeline Architecture

Trigger Mechanisms:

# Automatic triggers
on:
  push:
    branches: [main, develop]
    tags: ['v*.*.*']
  pull_request:
    branches: [main]

# Manual triggers
workflow_dispatch:
  inputs:
    test_type: [quick, full, build-only, lint-only]
    python_version: [3.8, 3.9, 3.10, 3.11, 3.12]
    create_artifacts: boolean

Pipeline Stages:

1. Code Quality: Trunk linting (Black + Ruff), pre-commit hooks
2. Testing: pytest with coverage, matrix testing across Python versions
3. Security: Dependency vulnerability scanning
4. Build: Clean package building, content verification
5. Release: PyPI publishing, Docker image, GitHub release

Release Process:

# Version bump
git tag v1.0.x -m "Release message"
git push origin v1.0.x

# Triggers automated pipeline:
# 1. Run full test suite
# 2. Build clean packages (excludes tests/, tools/, debug/)
# 3. Publish to PyPI as 'stockholm-finance'
# 4. Build multi-platform Docker image
# 5. Create GitHub release with artifacts
# 6. Send notifications

Testing Framework Architecture

Test Organization:

tests/
├── conftest.py                      # Shared fixtures and configuration
├── test_cache_simple.py             # 32 tests, comprehensive cache system
├── test_cache_background_fetcher.py # 12 tests, background integration
├── test_sentiment_analyzer.py       # 12 tests, core NLP functionality
├── test_policy_analyzer.py          # 13 tests, policy analysis
├── test_imports.py                  # 6 tests, import validation
└── README.md                        # Test documentation and usage guide

Key Features:

• 75 total tests with comprehensive cache system coverage
• Standard pytest approach (no custom test runners)
• Flat structure for simplicity and ease of navigation
• 44 cache tests covering the legendary enhancement system

Test Coverage Strategy:

• Unit Tests: 80%+ coverage for core components
• Integration Tests: Critical data flow paths
• Performance Tests: Cache efficiency, memory usage
• Regression Tests: Previous bug scenarios

Mock Strategy:

# External API mocking
@patch('yfinance.Ticker')
def test_with_mocked_yfinance(mock_ticker):
    mock_ticker.return_value.info = {'symbol': 'AAPL', 'price': 150.0}

# Cache mocking for isolated testing
@patch('src.data.cache_manager.CacheManager')
def test_without_cache(mock_cache):
    mock_cache.get.return_value = None  # Force cache miss

Dependency Management

Package Configuration:

• requirements.txt: Production dependencies only
• pyproject.toml: Complete package metadata and dev dependencies
• setup.py: Fallback configuration for older pip versions

Dependency Categories:

DEPENDENCIES = {
    'core': ['textblob', 'yfinance', 'requests', 'numpy'],
    'ui': ['textual', 'rich', 'textual_plotext'],
    'data': ['feedparser', 'python-dateutil', 'pytz'],
    'dev': ['pytest', 'pytest-cov', 'black', 'ruff'],
    'build': ['build', 'twine', 'wheel']
}

Version Pinning Strategy:

• Major versions pinned: Prevent breaking changes
• Minor versions flexible: Allow bug fixes and features
• Security updates: Automated dependency updates via Dependabot

Package Details

• PyPI Name: stockholm-finance (not stockholm - name conflict)
• Entry Point: Single command stockholm-finance (removed stockholm-dashboard)
• Python Support: 3.8+ with matrix testing
• Docker: Multi-platform builds pushed to GitHub Container Registry

Testing Framework

• 75 tests total: All passing with comprehensive coverage
• Cache system: 44 tests covering proactive refresh, intelligent warming, graceful degradation
• Core modules: 31 tests for sentiment analysis, policy analysis, imports
• Coverage: Cache manager (81%), overall (27% due to UI modules)
• Standard pytest: Simple, industry-standard testing approach
• Performance: < 3 seconds execution time for full test suite

Debugging and Troubleshooting

Common Issues and Solutions

Cache-Related Issues:

# Problem: Stale data showing in dashboard
# Solution: Clear specific cache type
python tools/cache_monitor.py --clear news

# Problem: High API usage despite caching
# Solution: Check cache hit rates
python tools/cache_monitor.py --stats

# Problem: Cache corruption errors
# Solution: Emergency cache clear
python tools/clear_all_cache.py

API Integration Issues:

# Problem: yfinance connection errors
# Debug: Test specific ticker
python tools/debug/debug_yfinance.py AAPL

# Problem: Rate limiting errors
# Debug: Check request patterns
python tools/cache_monitor.py --estimate

# Problem: Data validation failures
# Debug: Inspect raw API responses
python tools/debug/debug_time.py

Dashboard Issues:

# Problem: UI rendering problems
# Solution: Check terminal compatibility
python -c "import textual; print('Textual OK')"

# Problem: Slow dashboard performance
# Debug: Profile data loading
python main.py --verbose

# Problem: Keyboard shortcuts not working
# Solution: Check terminal focus and key bindings

Debug Tools and Scripts

Cache Monitoring:

tools/cache_monitor.py --stats      # Performance statistics
tools/cache_monitor.py --estimate   # API savings estimation
tools/cache_monitor.py --clear      # Interactive cache clearing

API Debugging:

tools/debug/debug_yfinance.py       # Deep yfinance API testing
tools/debug/debug_time.py           # Time-related debugging
tools/debug/investigate_dominion_loss.py  # Specific ticker investigation

Performance Analysis:

tools/debug/explain_earnings_trends.py    # Earnings data analysis
python main.py --verbose                  # Detailed execution logging
python -m cProfile main.py               # Performance profiling

Logging and Monitoring

Log Levels:

LOGGING_CONFIG = {
    'DEBUG': 'Detailed execution flow, cache operations',
    'INFO': 'Normal operations, API calls, user actions',
    'WARNING': 'Recoverable errors, rate limiting, stale data',
    'ERROR': 'API failures, data corruption, critical errors',
    'CRITICAL': 'System failures, unrecoverable errors'
}

Performance Metrics:

• API Call Tracking: Request counts, response times, error rates
• Cache Performance: Hit/miss ratios, storage usage, cleanup frequency
• Memory Usage: Peak memory, garbage collection frequency
• UI Responsiveness: Render times, user interaction latency

Error Recovery Strategies

Graceful Degradation:

def fetch_with_fallback(ticker):
    """Fetch data with multiple fallback strategies."""
    try:
        return primary_data_source(ticker)
    except APIError:
        try:
            return cached_data_source(ticker)
        except CacheError:
            return minimal_data_fallback(ticker)

Data Consistency Checks:

def validate_ticker_data(data):
    """Comprehensive data validation."""
    checks = [
        ('price', lambda x: x > 0, 'Price must be positive'),
        ('volume', lambda x: x >= 0, 'Volume cannot be negative'),
        ('timestamp', lambda x: x < time.time(), 'Timestamp cannot be future'),
    ]

    for field, validator, message in checks:
        if not validator(data.get(field)):
            raise ValidationError(f"{field}: {message}")

Important Notes

• Always use package managers (pip, not manual edits) for dependencies
• Pre-commit hooks will auto-fix most linting issues
• Release workflow only publishes to PyPI for non-draft, non-prerelease versions
• Dashboard is primary interface - no separate CLI analyzer
• Cache system is critical - don't bypass it, use cached_data_fetcher.py
• All external API calls should go through cached_data_fetcher.py
• UI components use Textual framework - follow their patterns
• Error handling should be graceful with fallback strategies
• Performance monitoring is built-in - use the debug tools
• Data validation is mandatory for all external data sources

📄 License

MIT License - see LICENSE file for details.

---

Built with ❤️ for financial analysis