Pre-release

This release is a pre-release and may not be stable for production use.

HarvestMan is a modular, extensible and flexible web crawler program cum framework written in pure Python. HarvestMan? can be used to download files from websites according to a number of customized rules and constraints. It can be used to find information from websites matching keywords or regular expressions.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for anandpillai from gravatar.com anandpillai

Credits

Author: Anand B Pillai

Maintainer: Lukasz Szybalski

License

GNU General Public License (GPL) (GPLv2)

Tags

crawler spider web-crawler web-bot robot data-mining python

Classifiers

Report project as malware

Release history Release notifications | RSS feed

This release
Pre-release

2.0.4betadev-r169 This release

Pre-release

1.4.6 final

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page