Memorious
The solitary and lucid spectator of a multiform, instantaneous and almost intolerably precise world.
-- Funes the Memorious, Jorge Luis Borges
memorious is a light-weight web scraping toolkit. It supports scrapers that
collect structured or un-structured data. This includes the following use cases:
- Make crawlers modular and simple tasks reusable
- Provide utility functions to do common tasks such as data storage, HTTP session management
- Integrate crawlers with the Aleph and FollowTheMoney ecosystem
- Get out of your way as much as possible
memorious is part of the OpenAleph suite but can be used standalone as well.
Design
When writing a scraper, you often need to paginate through through an index page, then download an HTML page for each result and finally parse that page and insert or update a record in a database.
memorious handles this by managing a set of crawlers, each of which
can be composed of multiple stages. Each stage is implemented using a
Python function, which can be reused across different crawlers.
The basic steps of writing a Memorious crawler:
- Make YAML crawler configuration file
- Add different stages
- Write code for stage operations (optional)
- Test, rinse, repeat
Documentation
The documentation for Memorious is available at
docs.investigraph.dev/lib/memorious.
Feel free to edit the source files in the docs folder and send pull requests for improvements.
To serve the documentation locally, run mkdocs serve
License and Copyright
memorious, (C) -2024 Organized Crime and Corruption Reporting Project
memorious, (C) 2025 Data and Research Center – DARC
memorious4, (C) 2026 Data and Research Center – DARC
memorious4 is licensed under the AGPLv3 or later license.
Prior to version 4.0.0, memorious was released under the MIT license.
Metadata
Release files for memorious4 4.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| memorious4-4.2.0.tar.gz | 73.6 kB | Without provenance | Details for memorious4-4.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| memorious4-4.2.0-py3-none-any.whl | Python 3 | none | any | Details for memorious4-4.2.0-py3-none-any.whl |
Total release size: 165.1 kB
Release files / memorious4-4.2.0.tar.gz
| Download URL | memorious4-4.2.0.tar.gz |
|---|---|
| Size | 73.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b43c138e88cec8c76dd0d5042051a0885e3a847d32f87a2be34c7ae8e8c82305
|
|
BLAKE2b-256 checksum How to use checksums |
0fe7e10ce3ef3ade7c71feb46410d811f27f1315a9588c9b854e03ca8cb08d28
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-38-generic
|
Release files / memorious4-4.2.0-py3-none-any.whl
| Download URL | memorious4-4.2.0-py3-none-any.whl |
|---|---|
| Size | 91.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c52c46e6487f786cef2c62dc4693e83ea3a14b0d6861dfbe59d32a946884e198
|
|
BLAKE2b-256 checksum How to use checksums |
73f68919713383debaa156184a315f636a6be663912d721872ba33c5b59c8ac6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.5.1 CPython/3.14.4 Linux/7.0.0-38-generic
|
Security for memorious4 4.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.