Downloads BGPStream info solely for research purposes
Project description
bgpstream_website_collector
Package Description
This package downloads information from bgpstream.com and stores it in a CSV
As a caveat, I wrote this a very long time ago when I was still an undergrad. I've cleaned it up a bit, but this has a long way to go before it's good code quality PRs welcome. Also, almost everything is type ignored because mypy spazzes out with bs4 Also, the tests could really use a good update
Usage
from a script:
from pathlib import Path
from typing import Any
from bgpstream_website_collector import BGPStreamWebsiteCollector
def main():
# Leave csv_path as None to not write CSV
csv_path: Path = Path.home() / "Desktop" / "bgpstream_website.csv"
rows: list[dict[str, Any]] = BGPStreamWebsiteCollector(csv_path=csv_path).run()
if __name__ == "__main__":
main()
From the command line:
bgpstream_website_collector
Installation
Install python and pip if you have not already.
Then run:
pip3 install pip --upgrade
pip3 install wheel
For production:
pip3 install bgpstream_website_collector
This will install the package and all of it's python dependencies.
If you want to install the project for development:
git clone https://github.com/jfuruness/bgpstream_website_collector.git
cd bgpstream_website_collector
pip3 install -e .[test]
pre-commit install
To test the development package: Testing
Testing
To test the package after installation:
cd bgpstream_website_collector
pytest bgpstream_website_collector
ruff bgpstream_website_collector
black bgpstream_website_collector
mypy bgpstream_website_collector
If you want to run it across multiple environments, and have python 3.10 and 3.11 installed:
cd bgpstream_website_collector
tox
Credits
Huge contributions to original version in lib_bgp_data to the testing from Tony Zheng
History
1.1.2 Fixed event date bug 1.1.1 Fixed start_time -> start_date type 1.1.0 Added option to pass None to csv_path 1.0.0 First major release
Development/Contributing
- Fork it!
- Create your feature branch:
git checkout -b my-new-feature
- Test it
- Run tox
- Commit your changes:
git commit -am 'Add some feature'
- Push to the branch:
git push origin my-new-feature
- Ensure github actions are passing tests
- Email me at jfuruness@gmail.com if it's been a while and I haven't seen it
License
BSD License (see license file)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file bgpstream_website_collector-1.1.2.tar.gz
.
File metadata
- Download URL: bgpstream_website_collector-1.1.2.tar.gz
- Upload date:
- Size: 13.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0eb360d5a03f9e2f3b90127e800511be3115d47e74531e45080cc45beb71a8c3 |
|
MD5 | 08957cfdf25424e0109a5d80e59d2d12 |
|
BLAKE2b-256 | e68a5b9e24fe0d8481df3c4c58bf2409a5995978f22dab91b2ce6920b470e28c |
File details
Details for the file bgpstream_website_collector-1.1.2-py3-none-any.whl
.
File metadata
- Download URL: bgpstream_website_collector-1.1.2-py3-none-any.whl
- Upload date:
- Size: 15.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.12
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 84b231e2749d89d20f79256b20729b7d60d81a4d02c1a5946285338cc7fedc83 |
|
MD5 | 0d0a9803fbc004ec0171a6c51765dc8b |
|
BLAKE2b-256 | 2401bfdaf102f8a0a79b62a4252a117c4716ed4c38baa7cae19cc9d9d5dfc14b |