Skip to main content

Logo

MIT License version-shield release-shield python-shield

Table of Contents
  1. About The Project
  2. Getting Started
  3. Usage
  4. Contributing
  5. Sponsors
  6. License
  7. Contact
  8. Acknowledgements

About The Project

Scrapera is a completely Chromedriver free package that provides access to a variety of scraper scripts for most commonly used machine learning and data science domains. Scrapera directly and asynchronously scrapes from public API endpoints, thereby removing the heavy browser overhead which makes Scrapera extremely fast and robust to DOM changes. Currently, Scrapera supports the following crawlers:

  • Images
  • Text
  • Audio
  • Videos
  • Miscellaneous

  • The main aim of this package is to cluster common scraping tasks so as to make it more convenient for ML researchers and engineers to focus on their models rather than worrying about the data collection process

    DISCLAIMER: Owner or Contributors do not take any responsibility for misuse of data obtained through Scrapera. Contact the owner if copyright terms are violated due to any module provided by Scrapera.

    Prerequisites

    Prerequisites can be installed separately through the requirements.txt file as below

    pip install -r requirements.txt
    

    Installation

    Scrapera is built with Python 3 and can be pip installed directly

    pip install scrapera
    

    Alternatively, if you wish to install the latest version directly through GitHub then run

    pip install git+https://github.com/DarshanDeshpande/Scrapera.git
    

    Usage

    To use any sub-module, you just need to import, instantiate and execute

    from scrapera.video.vimeo import VimeoScraper
    scraper = VimeoScraper()
    scraper.scrape('https://vimeo.com/191955190', '540p')
    

    For more examples, please refer to the individual test folders in respective modules

    Contributing

    Scrapera welcomes any and all contributions and scraper requests. Please raise an issue if the scraper fails at any instance. Feel free to fork the repository and add your own scrapers to help the community!
    For more guidelines, refer to CONTRIBUTING

    License

    Distributed under the MIT License. See LICENSE for more information.

    Sponsors

    Logo

    Contact

    Feel free to reach out for any issues or requests related to Scrapera

    Darshan Deshpande (Owner) - Email | LinkedIn

    Acknowledgements

    Metadata

    Release files for scrapera 1.1.3

    For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

    Source distribution (sdist)

    Source distribution for scrapera 1.1.3
    File Size Uploaded Provenance Actions
    scrapera-1.1.3.tar.gz 4.1 kB Without provenance Details for scrapera-1.1.3.tar.gz

    Built distribution (wheel)

    Table of built distributions (wheels) for scrapera 1.1.3
    File Interpreter ABI Platform Actions
    scrapera-1.1.3-py3-none-any.whl Python 3 none any Details for scrapera-1.1.3-py3-none-any.whl

    Total release size: 32.8 kB

    Release files / scrapera-1.1.3.tar.gz

    Download URL scrapera-1.1.3.tar.gz
    Size 4.1 kB
    Tags Source
    SHA-256 checksum
    How to use checksums
    985d28a499ea091eb9579f5514db0b34decb7c000a5a1a11215dec2271dd542f
    BLAKE2b-256 checksum
    How to use checksums
    28e600a182708780324a0639402181ade21db8046fefd2d4ec41e1f1daaf1b20
    Upload date
    Uploaded using Trusted Publishing?
    What is trusted publishing?
    No
    Uploaded via twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.7

    Release files / scrapera-1.1.3-py3-none-any.whl

    Download URL scrapera-1.1.3-py3-none-any.whl
    Size 28.7 kB
    Tags Python 3
    SHA-256 checksum
    How to use checksums
    502d98ca7578f2b4e935f3bf19adecc8e72c4091b3cc7494690e616739a5a0ae
    BLAKE2b-256 checksum
    How to use checksums
    25c61ae545436fcd4504edbb7ff9f3298d66e5c172d849f47686bc5c5337c387
    Upload date
    Uploaded using Trusted Publishing?
    What is trusted publishing?
    No
    Uploaded via twine/3.3.0 pkginfo/1.7.0 requests/2.25.1 setuptools/46.4.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.7.7

    Security for scrapera 1.1.3

    Provenance

    Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

    No provenance found

    None of the files in this release were uploaded with provenance metadata.

    A note on package security

    PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

    To learn more about protecting yourself from malicious actors, visit our package security documentation.

    Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page