Downloads BGPStream info solely for research purposes
Project description
bgpstream_website_collector
Package Description
This package downloads information from bgpstream.com and stores it in a CSV
As a caveat, I wrote this a very long time ago when I was still an undergrad. I've cleaned it up a bit, but this has a long way to go before it's good code quality PRs welcome. Also, almost everything is type ignored because mypy spazzes out with bs4 Also, the tests could really use a good update
Usage
from a script:
from pathlib import Path
from typing import Any
from bgpstream_website_collector import BGPStreamWebsiteCollector
def main():
# Leave csv_path as None to not write CSV
csv_path: Path = Path.home() / "Desktop" / "bgpstream_website.csv"
rows: list[dict[str, Any]] = BGPStreamWebsiteCollector(csv_path=csv_path).run()
if __name__ == "__main__":
main()
From the command line:
bgpstream_website_collector
Installation
Install python and pip if you have not already.
Then run:
pip3 install pip --upgrade
pip3 install wheel
For production:
pip3 install bgpstream_website_collector
This will install the package and all of it's python dependencies.
If you want to install the project for development:
git clone https://github.com/jfuruness/bgpstream_website_collector.git
cd bgpstream_website_collector
pip3 install -e .[test]
pre-commit install
To test the development package: Testing
Testing
To test the package after installation:
cd bgpstream_website_collector
pytest bgpstream_website_collector
ruff bgpstream_website_collector
black bgpstream_website_collector
mypy bgpstream_website_collector
If you want to run it across multiple environments, and have python 3.10 and 3.11 installed:
cd bgpstream_website_collector
tox
Credits
Huge contributions to original version in lib_bgp_data to the testing from Tony Zheng
History
1.1.2 Fixed event date bug 1.1.1 Fixed start_time -> start_date type 1.1.0 Added option to pass None to csv_path 1.0.0 First major release
Development/Contributing
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature
- Test it
- Run tox
- Commit your changes:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Ensure github actions are passing tests
- Email me at jfuruness@gmail.com if it's been a while and I haven't seen it
License
BSD License (see license file)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for bgpstream_website_collector-1.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0eb360d5a03f9e2f3b90127e800511be3115d47e74531e45080cc45beb71a8c3 |
|
MD5 | 08957cfdf25424e0109a5d80e59d2d12 |
|
BLAKE2b-256 | e68a5b9e24fe0d8481df3c4c58bf2409a5995978f22dab91b2ce6920b470e28c |
Hashes for bgpstream_website_collector-1.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84b231e2749d89d20f79256b20729b7d60d81a4d02c1a5946285338cc7fedc83 |
|
MD5 | 0d0a9803fbc004ec0171a6c51765dc8b |
|
BLAKE2b-256 | 2401bfdaf102f8a0a79b62a4252a117c4716ed4c38baa7cae19cc9d9d5dfc14b |