Skip to main content

Downloads

GraphScraper

GraphScraper is a Python 3 library that contains a base graph implementation designed to be turned into a web scraper for graph data. It has two major features:

1) The graph automatically manages a database (using either SQLAlchemy or Flask-SQLAlchemy) where it stores all the nodes and edges the graph has seen.

2) The base graph implementation provides hook methods that, if implemented, turn the graph into a web scraper.

Yet another graph implementation - why

There are many excellent graph libraries available for different purposes. I started implementing this one because i haven’t found a graph library that is dynamic (i don’t need the whole graph in memory - or on disk - before i start working with it), that can be used as a web scraper (to seamlessly load nodes and edges from some remote data source when that piece of data is needed) and that keeps all data (the graph) automatically up-to-date on the disk. GraphScraper aims to satisfy these requirements.

Examples

Besides the base graph implementation, the following working examples are also included in the library, that show you how you can implement and use an actual graph scraper:

  • igraphwrapper: Instead of web-scraping, this example is using an igraph graph instance as the “remote” source to scrape data from.

  • spotifyartist: This example is using the Spotify web API to load artists and edges are defined by Artist similarity.

Dependencies

If you wish to use one of the included graph implementations, then please read the corresponding module’s description for additional requirements.

Contribution

Any form of constructive contribution (feedback, features, bug fixes, tests, additional documentation, etc.) is welcome.

Metadata

Release files for graphscraper 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for graphscraper 0.5.0
File Size Uploaded
graphscraper-0.5.0.tar.gz 18.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for graphscraper 0.5.0
File Interpreter ABI Platform
graphscraper-0.5.0-py3-none-any.whl Python 3 none any Details

Total release size: 39.9 kB

Release files / graphscraper-0.5.0.tar.gz

Download URL graphscraper-0.5.0.tar.gz
Size 18.2 kB
Tags Source
SHA-256 checksum
How to use checksums
1fcc9581753ba0eab79a560703c601489b987d57732f5ae5b704ad73b95b5e8a
BLAKE2b-256 checksum
How to use checksums
daa26d8f5f11fce7805bccd5cc66b92827d086eedd85bc09a1dbee3f803950ec
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.6

Release files / graphscraper-0.5.0-py3-none-any.whl

Download URL graphscraper-0.5.0-py3-none-any.whl
Size 21.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a12d2683c9a6129889f8936f7681d523dcf8968ccd0239f9a0a97c2933cc8b20
BLAKE2b-256 checksum
How to use checksums
b7bc7a8bcbd2f53ac74f8847f3795c2cb8ec4d86a9c155d89b2fc6d8e8f944d7
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/42.0.2 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.6

Release history Release notifications | RSS feed

This release

0.5.0 This release

2 release files

0.4.0

1 release file

0.3.0

1 release file

0.2.0

1 release file

0.1.1

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page