Skip to main content

Downloads BGPStream info solely for research purposes

Project description

Python 3.10 Python 3.11 Tests

bgpstream_website_collector

Package Description

This package downloads information from bgpstream.com and stores it in a CSV

As a caveat, I wrote this a very long time ago when I was still an undergrad. I've cleaned it up a bit, but this has a long way to go before it's good code quality PRs welcome. Also, almost everything is type ignored because mypy spazzes out with bs4 Also, the tests could really use a good update

Usage

from a script:

from pathlib import Path
from typing import Any

from bgpstream_website_collector import BGPStreamWebsiteCollector


def main():
    # Leave csv_path as None to not write CSV
    csv_path: Path = Path.home() / "Desktop" / "bgpstream_website.csv"
    rows: list[dict[str, Any]] = BGPStreamWebsiteCollector(csv_path=csv_path).run()


if __name__ == "__main__":
    main()

From the command line:

bgpstream_website_collector

Installation

Install python and pip if you have not already.

Then run:

pip3 install pip --upgrade
pip3 install wheel

For production:

pip3 install bgpstream_website_collector

This will install the package and all of it's python dependencies.

If you want to install the project for development:

git clone https://github.com/jfuruness/bgpstream_website_collector.git
cd bgpstream_website_collector
pip3 install -e .[test]
pre-commit install

To test the development package: Testing

Testing

To test the package after installation:

cd bgpstream_website_collector
pytest bgpstream_website_collector
ruff bgpstream_website_collector
black bgpstream_website_collector
mypy bgpstream_website_collector

If you want to run it across multiple environments, and have python 3.10 and 3.11 installed:

cd bgpstream_website_collector
tox

Credits

Huge contributions to original version in lib_bgp_data to the testing from Tony Zheng

History

1.1.2 Fixed event date bug 1.1.1 Fixed start_time -> start_date type 1.1.0 Added option to pass None to csv_path 1.0.0 First major release

Development/Contributing

  1. Fork it!
  2. Create your feature branch: git checkout -b my-new-feature
  3. Test it
  4. Run tox
  5. Commit your changes: git commit -am 'Add some feature'
  6. Push to the branch: git push origin my-new-feature
  7. Ensure github actions are passing tests
  8. Email me at jfuruness@gmail.com if it's been a while and I haven't seen it

License

BSD License (see license file)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

bgpstream_website_collector-1.1.2.tar.gz (13.3 kB view details)

Uploaded Source

Built Distribution

File details

Details for the file bgpstream_website_collector-1.1.2.tar.gz.

File metadata

File hashes

Hashes for bgpstream_website_collector-1.1.2.tar.gz
Algorithm Hash digest
SHA256 0eb360d5a03f9e2f3b90127e800511be3115d47e74531e45080cc45beb71a8c3
MD5 08957cfdf25424e0109a5d80e59d2d12
BLAKE2b-256 e68a5b9e24fe0d8481df3c4c58bf2409a5995978f22dab91b2ce6920b470e28c

See more details on using hashes here.

File details

Details for the file bgpstream_website_collector-1.1.2-py3-none-any.whl.

File metadata

File hashes

Hashes for bgpstream_website_collector-1.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 84b231e2749d89d20f79256b20729b7d60d81a4d02c1a5946285338cc7fedc83
MD5 0d0a9803fbc004ec0171a6c51765dc8b
BLAKE2b-256 2401bfdaf102f8a0a79b62a4252a117c4716ed4c38baa7cae19cc9d9d5dfc14b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page