Skip to main content

Docs memorious4 on pypi PyPI Downloads PyPI - Python Version Python test and package pre-commit Coverage Status AGPLv3+ License Pydantic v2

Memorious

The solitary and lucid spectator of a multiform, instantaneous and almost intolerably precise world.

-- Funes the Memorious, Jorge Luis Borges

memorious is a light-weight web scraping toolkit. It supports scrapers that collect structured or un-structured data. This includes the following use cases:

  • Make crawlers modular and simple tasks reusable
  • Provide utility functions to do common tasks such as data storage, HTTP session management
  • Integrate crawlers with the Aleph and FollowTheMoney ecosystem
  • Get out of your way as much as possible

memorious is part of the OpenAleph suite but can be used standalone as well.

Design

When writing a scraper, you often need to paginate through through an index page, then download an HTML page for each result and finally parse that page and insert or update a record in a database.

memorious handles this by managing a set of crawlers, each of which can be composed of multiple stages. Each stage is implemented using a Python function, which can be reused across different crawlers.

The basic steps of writing a Memorious crawler:

  1. Make YAML crawler configuration file
  2. Add different stages
  3. Write code for stage operations (optional)
  4. Test, rinse, repeat

Documentation

The documentation for Memorious is available at docs.investigraph.dev/lib/memorious. Feel free to edit the source files in the docs folder and send pull requests for improvements.

To serve the documentation locally, run mkdocs serve

memorious, (C) -2024 Organized Crime and Corruption Reporting Project

memorious, (C) 2025 Data and Research Center – DARC

memorious4, (C) 2026 Data and Research Center – DARC

memorious4 is licensed under the AGPLv3 or later license.

Prior to version 4.0.0, memorious was released under the MIT license.

see NOTICE and LICENSE

Metadata

Release files for memorious4 4.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for memorious4 4.2.0
File Size Uploaded Provenance Actions
memorious4-4.2.0.tar.gz 73.6 kB Without provenance Details for memorious4-4.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for memorious4 4.2.0
File Interpreter ABI Platform Actions
memorious4-4.2.0-py3-none-any.whl Python 3 none any Details for memorious4-4.2.0-py3-none-any.whl

Total release size: 165.1 kB

Release files / memorious4-4.2.0.tar.gz

Download URL memorious4-4.2.0.tar.gz
Size 73.6 kB
Tags Source
SHA-256 checksum
How to use checksums
b43c138e88cec8c76dd0d5042051a0885e3a847d32f87a2be34c7ae8e8c82305
BLAKE2b-256 checksum
How to use checksums
0fe7e10ce3ef3ade7c71feb46410d811f27f1315a9588c9b854e03ca8cb08d28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-38-generic

Release files / memorious4-4.2.0-py3-none-any.whl

Download URL memorious4-4.2.0-py3-none-any.whl
Size 91.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
c52c46e6487f786cef2c62dc4693e83ea3a14b0d6861dfbe59d32a946884e198
BLAKE2b-256 checksum
How to use checksums
73f68919713383debaa156184a315f636a6be663912d721872ba33c5b59c8ac6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-38-generic

Security for memorious4 4.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page