Downloads BGPStream info solely for research purposes
Project description
bgpstream_website_collector
Package Description
This package downloads information from bgpstream.com and stores it in a CSV
As a caveat, I wrote this a very long time ago when I was still an undergrad. I've cleaned it up a bit, but this has a long way to go before it's good code quality PRs welcome. Also, almost everything is type ignored because mypy spazzes out with bs4 Also, the tests could really use a good update
Usage
from a script:
from pathlib import Path
from typing import Any
from bgpstream_website_collector import BGPStreamWebsiteCollector
def main():
# Leave csv_path as None to not write CSV
csv_path: Path = Path.home() / "Desktop" / "bgpstream_website.csv"
rows: dict[str, Any] = BGPStreamWebsiteCollector(csv_path=csv_path).run()
if __name__ == "__main__":
main()
From the command line:
bgpstream_website_collector
Installation
Install python and pip if you have not already.
Then run:
pip3 install pip --upgrade
pip3 install wheel
For production:
pip3 install bgpstream_website_collector
This will install the package and all of it's python dependencies.
If you want to install the project for development:
git clone https://github.com/jfuruness/bgpstream_website_collector.git
cd bgpstream_website_collector
pip3 install -e .[test]
pre-commit install
To test the development package: Testing
Testing
To test the package after installation:
cd bgpstream_website_collector
pytest bgpstream_website_collector
ruff bgpstream_website_collector
black bgpstream_website_collector
mypy bgpstream_website_collector
If you want to run it across multiple environments, and have python 3.10 and 3.11 installed:
cd bgpstream_website_collector
tox
Credits
Huge contributions to original version in lib_bgp_data to the testing from Tony Zheng
History
1.1.0 Added option to pass None to csv_path 1.0.0 First major release
Development/Contributing
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature
- Test it
- Run tox
- Commit your changes:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Ensure github actions are passing tests
- Email me at jfuruness@gmail.com if it's been a while and I haven't seen it
License
BSD License (see license file)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for bgpstream_website_collector-1.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9cd8d64c2ff0d75acfef4194ca4f5a29606c7c0ce4e01a06c8c3661b733c7d9 |
|
MD5 | fdf4dca884ef67b664b70f7be46d214b |
|
BLAKE2b-256 | 415e3b0cff518a71bb533191483164dbb98937b18cfc469b7c2e0afbfe766205 |
Hashes for bgpstream_website_collector-1.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3c578e6917f36af47a4702396040d8322fb73780909dca2e3e508a2e2ab596f1 |
|
MD5 | 37a18cb3b00a57e70854e01b19aaa9c5 |
|
BLAKE2b-256 | a23d5730082e595001c5a49656ff8e5c45b6901755afca642b52c10c1eea91a5 |