Quick Start

# Install
pip install oddsharvester

# Or clone & setup with uv
git clone https://github.com/jordantete/OddsHarvester.git && cd OddsHarvester
pip install uv && uv sync

# Scrape upcoming football matches
oddsharvester upcoming -s football -d 20250301 -m 1x2 --headless

# Scrape historical Premier League odds
oddsharvester historic -s football -l england-premier-league --season 2024-2025 -m 1x2 --headless

# Snapshot odds for matches in play right now
oddsharvester live -s tennis -m match_winner --headless

# Scrape community data (top predictions here; also --user profiles and --match-url votes)
oddsharvester community -s football --headless

Features

Supported Sports & Markets

Umbrella tokens (football): over_under and asian_handicap are umbrella market tokens — pass either as --market and it expands at scrape time to every line OddsPortal actually renders for that match (e.g. over_under_1_5_market, over_under_2_5_market, …), instead of listing each line by hand.

Cricket: OddsPortal does not currently publish a per-bookmaker odds table for cricket, so cricket scraping returns match metadata (teams, league, score, result) with an empty odds list. The home_away market is wired and will populate if OddsPortal adds cricket odds.

100+ leagues supported across all sports: Premier League, La Liga, Serie A, NBA, NFL, MLB, NHL, ATP/WTA Grand Slams, and many more.

CLI Usage

OddsHarvester has four commands: upcoming, historic, live, and community. They share most options, with a few command-specific ones.

oddsharvester upcoming

Scrape odds for upcoming matches — by date, by league, or by specific match URL.

# By date
oddsharvester upcoming -s football -d 20250301 -m 1x2 --headless

# By league (scrapes all upcoming matches for that league)
oddsharvester upcoming -s football -l england-premier-league -m 1x2,btts --headless

# Multiple leagues
oddsharvester upcoming -s football -l england-premier-league,spain-laliga -m 1x2 --headless

# Specific match URLs (repeat the flag; works for past matches too)
oddsharvester upcoming -s football --match-link "https://www.oddsportal.com/football/..." -m 1x2

# Preview mode (faster — best/highest odds only, no individual bookmakers)
oddsharvester upcoming -s football -d 20250301 -m over_under --preview-only --headless

# Only matches kicking off within the next 6 hours (fewer requests)
oddsharvester upcoming -s football -l england-premier-league -m 1x2 --kickoff-within-hours 6 --headless

# Collect links plus kickoff only (fixture plan; no odds scraped)
oddsharvester upcoming -s football -d 20250301 --links-only -f csv -o upcoming_links.csv

oddsharvester historic

Scrape historical odds and results for past seasons.

# Single league & season
oddsharvester historic -s football -l england-premier-league --season 2022-2023 -m 1x2 --headless

# Current season
oddsharvester historic -s football -l england-premier-league --season current -m 1x2 --headless

# Limit pagination
oddsharvester historic -s football -l england-premier-league --season 2022-2023 -m 1x2 --max-pages 3 --headless

# Output as CSV
oddsharvester historic -s football -l england-premier-league --season 2024-2025 -m 1x2 -f csv -o premier_league_odds --headless

# Umbrella market — expands to every Over/Under line rendered on the page
oddsharvester historic -s football -l england-premier-league --season 2023-2024 --market over_under -f csv

oddsharvester live

Take a one-shot snapshot of matches currently in play, with per-bookmaker in-play odds.

# Every live match for a sport
oddsharvester live -s tennis -m match_winner --headless

# Restricted to one league
oddsharvester live -s football -l england-premier-league -m 1x2 --headless

# Re-sample one known match without reloading the listing
oddsharvester live -s football --match-link "https://www.oddsportal.com/football/..." -m 1x2 --headless

# Collect live match links only
oddsharvester live -s football --links-only -o live_links.json --headless

Each record carries the usual match metadata plus live context: live_period (the period marker exactly as the site displays it, so it is sport-specific: 4' or Half-time in football, 1st Set in tennis, 9th Inning in baseball), live_score_home, live_score_away, live_score_raw (keeps compound formats such as 0:1 (3:6, 4:2)), and scraped_at_utc, which is what makes a series of snapshots comparable.

Notes:

• Zero live matches is a normal outcome: the command prints a message and exits 0 without writing a file.
• Matches that end between listing and scrape are dropped, so a snapshot only ever contains genuinely live matches.
• In-play bookmaker coverage is thinner than pre-match (often 2 to 4 bookmakers instead of 15 to 20) and varies by region.
• --odds-history and --period are rejected: the in-play view exposes neither.
• For repeated sampling, schedule the command externally (cron or similar). Keep at least 60 seconds between snapshots, and prefer --match-link to re-sample a known match without re-reading the listing. The command itself never refreshes a page, which keeps its request profile identical to upcoming.

oddsharvester community

Scrape OddsPortal Community data. community has three mutually-exclusive modes; exactly one is required:

• Top predictions (--sport): the most-voted community picks for the next 7 days.
• User profile (--user <username>): a tipster's stats, monthly performance and recent predictions.
• Match community votes (--match-url <url>): per-market community vote volume for a single match.

Top predictions (--sport)

# Top predictions for a sport
oddsharvester community -s football --headless

# Write to a named JSON file
oddsharvester community -s football -f json -o top_predictions.json --headless

Each record contains the match (home_team, away_team, match_url, kickoff, plus the raw kickoff_text label kept as fallback when the date token fails to parse), the league (sport, country, league), the voted market, best odds per outcome (odds), the community vote split (community_votes_pct), and scraped_at.

• OddsPortal surfaces ~10 picks per sport (no pagination) with rounded percentages.
• Pre-match only: OddsPortal drops community data from finished-match pages, so build longitudinal datasets by scraping while matches are still upcoming.

User profile (--user <username>)

oddsharvester community --user BLAPRO --headless

Emits one record: header (username, roi_pct, member_since, country, privacy), the monthly statistics table (month, total_predictions, won, lost, plus_minus, roi_pct, incl. a Total row), and the rendered predictions batch. Each prediction has market, home_team/away_team, score (when finished), and a positional outcomes list of {odds, community_pct, picked} plus pick_odds. Most profiles are private: a private profile returns the header only (privacy: "private", empty stats/predictions) and exits 0.

Match community votes (--match-url <url>)

oddsharvester community --match-url "https://www.oddsportal.com/football/h2h/.../" --headless

Emits one record with per-market community vote volume: markets[] of {market, scope, handicap, betting_type_id, scope_id, total_votes, outcome_counts}, most-voted first, plus top_community_pick. Pre-match only (OddsPortal drops community data from finished matches).

Limitations: --match-url outcome vote counts are unlabeled: OddsPortal obfuscates the per-outcome ids, so only per-market volume, the count distribution, and the single aggregate pick are recoverable. --user captures the first rendered predictions batch (no deep pagination) and does not emit per-prediction win/loss (use the monthly stats table).

oddsharvester team

Scrape team metadata from OddsPortal team pages: identity, venue, coach and recent form. One record per team.

# By team id, as it appears in an OddsPortal URL
oddsharvester team --team lId4TMwf --headless

# Several teams at once, by id or by pasted team page URL, into a spreadsheet
oddsharvester team --team lId4TMwf,WGt8En5I --headless -f csv -o teams.csv

# From a file holding one id or URL per line
oddsharvester team --teams-file my_teams.txt --headless -o teams.json

Each record carries team_id, name, full_name, list_name, sport, country, town, venue, coach, tournament, logo_url, the Last 6 Games block (form, over_2_5_pct, btts_pct, avg_goals_scored, avg_goals_conceded), the canonical team_url and scraped_at.

list_name is the short name OddsPortal prints in match lists, which is the one to join on: a team can appear as "Barcelona FC" on its own page and "Barcelona" in a fixture list. It is read from the recent-form block, so it survives a page that shows no odds rows at all.

Limitations: country, town and venue are simply absent for smaller teams, and tournament only appears on fixture rows, which your IP's selected bookmakers may leave empty. A wrong team id is reported as a failure rather than written as a blank record: the page it returns builds its heading from the URL and otherwise looks valid.

CLI Options Reference

Core Options

--match-link usage: --sport is still required. Prefer upcoming over historic for arbitrary match URLs: match links bypass the listing pages entirely, so upcoming also works for matches already played, while historic would additionally demand a --season it never uses. For large link sets (a --links-only output, re-running failures), prefer --match-links-file: a pasted command line gets silently truncated by the terminal past a few hundred URLs.

upcoming only: --date is required unless --league or --match-link is provided. --date and --league can be combined to filter the league's upcoming matches down to a specific calendar day. When combining both, the reference timezone for resolving the date is --timezone if provided, otherwise UTC. --kickoff-within-hours N keeps only matches starting within N hours from now; the filter runs during link collection, so far-off matches are never visited. It pairs with the default upcoming-only behaviour to bound the window on both sides, and uses --timezone (else UTC) as the reference clock. Combined with --links-only, each row also carries kickoff_utc, so a scheduler can plan a day of fixtures from one listing request instead of re-fetching the listing on every cycle.

historic only:

live only: no --date and no --season; the command always reads whatever is in play at the moment it runs. --league accepts at most one slug. --odds-history and --period are rejected outright, because the in-play view exposes neither. A --match-link given in classic form is normalized to its in-play URL automatically, so either form works. When every match fails to scrape the command exits non-zero, which is what lets a scheduled sampler tell a blocked run apart from a genuinely empty one.

Output Options

Breaking change: every output row now carries a season column. For odds rows it is inserted directly after match_date; --links-only rows have no match_date and carry season alongside the other link fields instead. It holds the scraped season for historic and is empty for upcoming and --match-link runs. Appending to a file produced by an earlier version yields a file with two different column layouts, so start a new output file rather than appending across the upgrade.

Breaking change: upcoming --links-only rows now carry a kickoff_utc column, appended at the end. Appending to a file produced by an earlier version yields a file with two different column layouts, so start a new output file rather than appending across the upgrade. historic and live links-only rows are unchanged.

Browser & Scraping Options

Proxy Options

League listings always load on the first proxy's browser context; only match pages rotate across the configured proxies.

Tip: For best results, match --locale and --timezone to your proxy's region.

Multi-proxy example — spread scraping across three proxies with embedded credentials:

oddsharvester historic --sport football --league england-premier-league --season 2013-2014 \
  --market 1x2 --concurrency 6 \
  --proxy-url http://user:pass@p1.example.com:8000 \
  --proxy-url http://user:pass@p2.example.com:8000 \
  --proxy-url http://user:pass@p3.example.com:8000

Matches are dispatched round-robin across the proxies; a proxy that fails 3 times in a row (navigation/rate-limit errors) is dropped from rotation and the run continues on the survivors.

Advanced Options

Two-pass workflow: collect links, then scrape

For large runs it can be safer to collect all match links first, then scrape odds per link and re-run only the failures (see issue #75):

# Pass 1 - collect the season's match links (no odds scraped)
oddsharvester historic -s football -l england-premier-league --season 2022-2023 \
    --links-only -f csv -o links.csv

# Pass 2 - scrape odds from the collected links (--append fills recovered failures)
tail -n +2 links.csv | cut -d, -f1 | sort -u > links.txt
oddsharvester upcoming -s football -m 1x2 -f csv -o odds.csv --append \
    --match-links-file links.txt

On upcoming, the same pass 1 doubles as a fixture plan: one listing request returns every match of the day with its kickoff, so a scheduler can decide offline which matches are close enough to be worth scraping.

# Plan the day's fixtures: links plus kickoff, no odds scraped
oddsharvester upcoming -s football -d 20260731 --links-only -f csv -o upcoming_links.csv

Output rows contain match_link, sport, league, and season (date and kickoff_utc for upcoming; live emits neither), in the site's listing order. kickoff_utc holds the match's own kickoff in UTC, in the same shape as match_date (2026-07-31 18:30:00 UTC), and is empty when the listing exposes no parseable kickoff. Under the default upcoming-only behaviour that means the date header could not be read, the row had no time-item element, or its clock text was not HH:MM; with --include-started, live rows still come back empty, since their clock is replaced by a period marker; postponed, cancelled, and most finished rows keep the clock instead, giving a populated kickoff_utc with the originally scheduled kickoff. Options that only affect odds scraping (--market, --period, --odds-history, --preview-only, --target-bookmaker, --bookies-filter) are ignored when --links-only is set. --links-only cannot be combined with --match-link.

Bulk scraping: multiple leagues, multiple seasons

--season and --league both accept comma-separated lists. historic scrapes every combination as the cartesian product, league outer and season inner; listings run up to --concurrency at once, but output still stays grouped and deterministic per combo. --max-pages applies per combo, not per run.

# Several seasons of one league
oddsharvester historic --sport football --league england-premier-league \
    --season 2020-2021,2021-2022,2022-2023 --links-only --format csv --output links.csv

# Cartesian product: every league by every season
oddsharvester historic --sport football \
    --league england-premier-league,spain-laliga \
    --season 2021-2022,2022-2023 --links-only

# A league that changed season format mid-history: pass both formats and
# let the invalid pairs report zero
oddsharvester historic --sport football --league russia-premier-league \
    --season 2010,2010-2011,2011,2011-2012 --links-only

No pre-filtering is attempted to figure out which (league, season) pairs are valid before scraping them; a wrong-format pair (e.g. 2010 for a league that only ever used 2010-2011) is simply scraped and returns zero links. That is normal, not an error: OddsPortal returns HTTP 200 for a dead season URL, so the only way to tell a valid combo from an invalid one is to scrape it and count the results (see docs/agentic-gotchas.md §15). When more than one combo runs, an end-of-run table lists each league/season pair with its count, then reports how many combos returned nothing and how many errored. A zero-count combo does not affect the exit code; only errored combos are worth re-running.

Local kickoff time

--local-kickoff adds two fields to each record: venue_timezone (the venue's IANA timezone id) and match_date_venue_local (the kickoff converted to that timezone, with an explicit offset), e.g. 2022-05-01 16:00:00 BST+0100. match_date stays UTC; the two fields are additive and only appear when the flag is set.

Resolution is best-effort from the record's venue country/town. Single-timezone countries resolve by country; USA, Canada, Mexico, Brazil, Russia, and Australia resolve by host city instead. A venue that can't be resolved gets null for both fields.

Not compatible with --links-only (no match pages are visited, so there's no venue to resolve). Distinct from --timezone, which sets the browser's context timezone and does not affect the output fields.

If you --append onto an existing CSV file, the header is frozen on the first write, so start a fresh file when you turn the flag on.

Streaming results while the run is in progress

By default a run collects every match and writes the whole batch at the end, so a 10-match run hands you nothing for 10 minutes. --stream-ndjson emits each match on stdout as a single JSON line the moment that match is done, letting a downstream process start work while the rest is still being scraped:

oddsharvester upcoming -s football -d 20260918 -m 1x2 --stream-ndjson --headless \
    | while read -r line; do echo "$line" | jq -r '.home_team + " vs " + .away_team'; done

Each line is exactly the record that --output would have written, so a consumer parses the same shape either way. Logs, the run summary and the failed-URL list all go to stderr, leaving stdout carrying nothing but NDJSON.

Without --output, no result file is written at all — the stream is the output. Pass --output to keep a batch file as well; --format then governs that file only, the stream stays NDJSON.

Failures are not streamed: they stay visible on stderr and in the exit code. Available on upcoming, historic and live; not compatible with --links-only, which never visits match pages. A run that spans several leagues or seasons streams every match into one flat sequence with no marker between combos.

Blocked odds

OddsPortal strikes through a price when that bookmaker has stopped offering the bet. Each per-bookmaker odds record then carries a blocked_outcomes field listing which outcomes are struck through, using the same labels as the odds themselves:

{
  "bookmaker_name": "Unibet.fr",
  "1": "1.32",
  "X": "4.55",
  "2": "6.10",
  "blocked_outcomes": ["1", "X", "2"]
}

The labels are the market's own, so they differ per market: 1 / X / 2 for 1X2, odds_over / odds_under for Over/Under, and so on.

The field is always collected, with no flag to enable, and is omitted entirely when nothing is blocked, so records for available odds are unchanged. Odds values are kept exactly as rendered: a struck-through price is still the last price that bookmaker showed. A bookmaker with no price at all renders - and is not flagged, so "no odds" and "blocked" stay distinguishable.

Environment Variables

All CLI options can be set via environment variables — useful for Docker or CI/CD.

export OH_SPORT=football
export OH_HEADLESS=true
export OH_PROXY_URL=http://proxy.example.com:8080

oddsharvester upcoming -d 20250301 -m 1x2

Installation

With pip (from PyPI)

pip install oddsharvester

From source (with uv)

git clone https://github.com/jordantete/OddsHarvester.git
cd OddsHarvester
pip install uv
uv sync

python3 -m venv .venv
source .venv/bin/activate    # Unix/macOS
# .venv\Scripts\activate     # Windows

pip install . --use-pep517
# or: poetry install

Verify installation:

oddsharvester --help

Docker

# Build
docker build -t odds-harvester:local .

# Run (CLI args are appended to the ENTRYPOINT `python3 -m oddsharvester`)
docker run --rm odds-harvester:local upcoming -s football -d 20250301 -m 1x2 --headless

# Run and keep the JSON output on the host (mount a volume + use -o)
# On macOS+colima, prefer a path under $HOME (e.g. $PWD); /tmp is not shared by default.
docker run --rm -v "$PWD/_docker_out:/out" odds-harvester:local \
  upcoming -s football -d 20250301 -m 1x2 --headless -o /out/result.json

# Or with environment variables
docker run --rm \
  -e OH_SPORT=football \
  -e OH_HEADLESS=true \
  odds-harvester:local upcoming -d 20250301 -m 1x2

Contributing

Contributions are welcome! Submit an issue or pull request. Please follow the project's coding standards and include clear descriptions for any changes.

License

MIT License

Disclaimer

This package is intended for educational purposes only. The author is not affiliated with or endorsed by oddsportal.com. Use responsibly and ensure compliance with their terms of service and applicable laws.