athlinks-races: web scraper for race results hosted on Athlinks
Introduction
athlinks provides the RaceSpider class.
This spider crawls through all results pages from a race hosted on athlinks.com, building and following links to each athlete's individual results page, where it collects their split data. It also collects some metadata about the race itself.
By default, the spider returns one race metadata object (RaceItem), and one AthleteItem per participant.
Each AthleteItem consists of some basic athlete info and a list of RaceSplitItem containing data from each
split they recorded.
How to use this package
Using uv
If you have installed uv, is as simple as this:
uvx --from athlinks-races athlinks_races_cli --tui --race_url https://www.athlinks.com/event/382111/results/Event/1093108/Results
python scripts
Scrapy can be operated entirely from python scripts. See the scrapy documentation for more info.
Installation
The package is available on PyPi. In your own uv project:
uv add athlinks-races
Or, inside an existing virtual environment:
uv pip install athlinks-races
Example usage
A demo script is included in this repo. It has plenty of features but is also monolithic on purpose.
"""
Demonstrate the available classes.
You can run as `uv run python -m athlinks.demo`
"""
from scrapy.crawler import CrawlerProcess
from athlinks import RaceSpider, AthleteItem, RaceItem
def main():
# Make settings for two separate output files: one for athlete data,
# one for race metadata.
settings = {
'FEEDS': {
# Athlete data. Inside this file will be a list of dicts containing
# data about each athlete's race and splits.
'athletes.json': {
'format': 'json',
'overwrite': True,
'item_classes': [AthleteItem],
},
# Race metadata. Inside this file will be a list with a single dict
# containing info about the race itself.
'metadata.json': {
'format': 'json',
'overwrite': True,
'item_classes': [RaceItem],
},
}
}
process = CrawlerProcess(settings=settings)
# Crawl results for the 2022 Leadville Trail 100 Run
process.crawl(RaceSpider, 'https://www.athlinks.com/event/33913/results/Event/1018673/')
process.start()
if __name__ == "__main__":
main()
If you clone this repo and run uv sync --all-groups then you can run as
uv run athlinks_races_cli
Then you can build the distributions (sdist and wheel) to install locally if needed:
uv build
Command line
Alternatively, you may clone this repo for use like a typical Scrapy project that you might create on your own.
Installation
git clone https://github.com/josevnz/athlinks-races
cd athlinks-races
uv sync --all-groups
uv sync creates the virtual environment in .venv and installs the Python version pinned in
.python-version (3.14) if it is not already available.
Example usage
Run a RaceSpider, few races with different years:
cd athlinks-races
uv run scrapy crawl race -a url=https://www.athlinks.com/event/33913/results/Event/1018673 -O $HOME/1018673.json
uv run scrapy crawl race -a url=https://www.athlinks.com/event/382111/results/Event/1093108 -O $HOME/1093108.json
uv run scrapy crawl race -a url=https://www.athlinks.com/event/382111/results/Event/1062909 -O $HOME/1093108.json
Or the newer athlinks_races_cli, with the --tui argument:
(athlinks-races) [josevnz@dmaf5 athlinks-races]$ uv run athlinks_races_cli --tui
2025-01-12 14:56:31 [scrapy.middleware] INFO: Enabled item pipelines:
[]
2025-01-12 14:56:31 [scrapy.core.engine] INFO: Spider opened
2025-01-12 14:56:31 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2025-01-12 14:56:31 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023
2025-01-12 14:56:31 [asyncio] DEBUG: Using selector: EpollSelector
Dependencies
All that is required is Scrapy and Textual (and its dependencies).
Testing
uv run pytest tests/test_athlinks.py
Example session:
(athlinks-races) [josevnz@dmaf5 athlinks-races]$ uv run pytest /home/josevnz/athlinks-races/tests/test_athlinks.py
============================================================= test session starts =============================================================
platform linux -- Python 3.11.6, pytest-8.3.3, pluggy-1.5.0
rootdir: /home/josevnz/athlinks-races
configfile: pyproject.toml
collected 6 items
tests/test_athlinks.py ...... [100%]
============================================================== 6 passed in 0.33s ==============================================================
License
This project is licensed under the MIT License. See LICENSE file for details.
Contact
You can get in touch with me here:
- GitHub: https://github.com/josevnz
Original Author
If you want to take a look at the original project. He is not in charge of this forked version.
- GitHub: github.com/aaron-schroeder
Metadata
Release files for athlinks-races 0.0.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| athlinks_races-0.0.9.tar.gz | 19.3 kB | Without provenance | Details for athlinks_races-0.0.9.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| athlinks_races-0.0.9-py3-none-any.whl | Python 3 | none | any | Details for athlinks_races-0.0.9-py3-none-any.whl |
Total release size: 43.3 kB
Release files / athlinks_races-0.0.9.tar.gz
| Download URL | athlinks_races-0.0.9.tar.gz |
|---|---|
| Size | 19.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3261e7ccdbe232a3cf0cd7ce1a54a3440274e5a0446aa81a8fc0b54f23d32d08
|
|
BLAKE2b-256 checksum How to use checksums |
70e9d3a5704d1f184efdb37cae7fc289022d88a04ebfd4e514f5b983364c1de5
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Fedora Linux","version":"37","id":"","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / athlinks_races-0.0.9-py3-none-any.whl
| Download URL | athlinks_races-0.0.9-py3-none-any.whl |
|---|---|
| Size | 24.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
75760fdaa6575e115cd720b082f665d67094fe3423ecdcf76d0ec44aed023bbf
|
|
BLAKE2b-256 checksum How to use checksums |
3074d16718998694abdaad966b0983e8e0ff9923378d0eebdff99064ae7ec456
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.23 {"installer":{"name":"uv","version":"0.12.23","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Fedora Linux","version":"37","id":"","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for athlinks-races 0.0.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.