A generic specification for mirroring data
Project description
mirrorlib
mirrorlib is a flat, generic specification for tracking urls and files together. The strength of mirrorlib is its simplicity and ability to track any type of data. It is designed to make data verification and integrity easy and provide all the data upfront to manage your mirror. mirrorlib is not designed to be the fastest mirror specification, but to be as portable and simple as possible. This allows many developers to quickly develop on top of the code and have the flexibility to use the data as they please.
mirrorlib was inspired by RedoxOS and their philosophy of "everything is a URL". In mirrorlib everything is a URI that is tracked together in one place.
mirrorlib is not specific to any type of database and can be implemented using any language, you could even implement it as a JSON file if you wanted to. The reference implementation is currently implemented using python and sqlite. It's not required, but all data is usually stored in a single table called 'mirror'. A mirror contains only URIs and their associated metadata as text.
Getting Started
pip install mirrorlib
Documentation
Golden rules
- Everything is a URI
- Use text as much as possible
- Keep data simple
Contributing
TODO
The specification and reference implementation is still being developed and any suggestions and issues are welcome.
The longterm goal is to implement the reference specification in Rust and use Python bindings instead. For the time being
History
mirrorlib was originally created to be used in another archiving project, youmirror. Youmirror is a tool developed in early 2022 for downloading and archiving media, originally Youtube videos. However, I'm always looking for ways that components of larger projects can be broken down into modules that may be helpful more broadly. I am hoping that mirrorlib will be used to archive all kinds of data and inspire even better projects in the future!
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mirrorlib-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 227da540956660728de9c4e0499510c06e8eb33da739a0cbaeecf603eb723bb9 |
|
MD5 | 8db28a45ab145c93c34817e7e79789f6 |
|
BLAKE2b-256 | d0e7d3d208e684a300f0714d561cb0ac6b875e8b46ef002fc03f6cf3710b2d86 |