Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc Optimized for use with FastAPI/Pydantic.
Project description
msaFilesystem - Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc
Optimized for use with FastAPI/Pydantic.
Alpha/EarlyStage !!!
Documentation: msaFilesystem Documentation (https://msaFilesystem.u2d.ai/)
Features
- Abstract Filesystem: Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc. Based on: PyFilesystem2
- App Filesystems: Manage filesystems in platform-specific application directories. These classes abstract away the different requirements for user data across platforms.
- FTP Filesystem: A FTP (File Transport Protocol) Filesystem. Optionally, the connection can be made securely via TLS. This is known as FTPS, or FTP Secure.
- Memory Filesystem: A filesystem that stored in memory. Memory filesystems are useful for caches, temporary data stores, unit testing, etc.
- Mount Filesystem: A Mount FS is a virtual filesystem which can seamlessly map sub-directories on to other filesystems.
- Multi Filesystem: A MultiFS is a filesystem composed of a sequence of other filesystems, where the directory structure of each overlays the previous filesystem in the sequence.
- OS Filesystem: Manage the filesystem provided by your OS. In essence, an OSFS is a thin layer over the io and os modules of the Python standard library.
- Sub Filesystem: Manage a directory in a parent filesystem. A SubFS is a filesystem object that maps to a sub-directory of another filesystem.
- Tar Filesystem: Read and write tar files.
- Temporary Filesystem: Manage filesystems in temporary locations. A temporary filesytem is stored in a location defined by your OS (/tmp on linux). The contents are deleted when the filesystem is closed.
- Zip Filesystem: Read and write zip files.
- SMB Filesystem: A filesystem over SMB.
- WebDAV Filesystem: A filesystem for WebDAV.
- Azure Datalake & S3FS Filesystem: A filesystem for Azure Datalake storage & S3FS.
- Google Cloud Storage (GCS) Filesystem: A filesystem for Google Cloud Storage (GCS). With GCSFS, you can interact with Google Cloud Storage as if it was a regular filesystem.
- Google Drive Filesystem: A filesystem for Google Drive. Interact with Google Drive as if it was a regular filesystem.
- Dropbox Filesystem: A filesystem for Dropbox.
- OneDrive Filesystem: A filesystem for Dropbox.
- YouTube Videos and Playlists Filesystem: A filesystem for YouTube Videos and Playlists.
- External Filesystems: See the following wiki page for a list of filesystems not in the core library, and community contributed filesystems.
Main Dependencies
- fs~=2.4.16: Module that provides a common interface to any filesystem
- six~=1.16.0: Python 2 and 3 compatibility utilities
- pysmb~=1.2.8: SMB/CIFS library
- fs.webdavfs~=0.4.2: WebDAV support
- fs-dlk~=0.1.3: Azure Datalake support
- fs-s3fs~=1.1.1: Amazon S3 filesystem support
- fs-gcsfs~=1.5.1: Google Cloud Storage (GCS) support
- fs.googledrivefs~=2.3.0: Google Drive support
- fs.dropboxfs~=0.2.2.post2: Dropbox support
- fs.onedrivefs~=1.1.1: OneDrive support
- fs.youtube~=0.3.1: Youtube support
- fs.smbfs~=1.0.5: SMB support
License Agreement
msaFilesystem
is based onMIT
open source and free to use, it is free for commercial use, but please show/list the copyright information about msaFilesystem somewhere.
How to create the documentation
We use mkdocs and mkdocsstring. The code reference and nav entry get's created virtually by the triggered python script /docs/gen_ref_pages.py while mkdocs
serve
or build
is executed.
Requirements Install for the PDF creation option:
PDF Export is using mainly weasyprint, if you get some errors here pls. check there documentation. Installation is part of the msaFilesystem, so this should be fine.
We can now test and view our documentation using:
mkdocs serve
Build static Site:
mkdocs build
Build and Publish
Build:
python setup.py sdist
Publish to pypi:
twine upload dist/*
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file msafilesystem-0.0.8.tar.gz
.
File metadata
- Download URL: msafilesystem-0.0.8.tar.gz
- Upload date:
- Size: 8.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.9.18 Linux/6.2.0-1012-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7d44f0556a718a537ce0d5275f859e94b9e5e2c7d121c3aaf62e248a8d138f28 |
|
MD5 | f06046e0e605e206bbd609c4feb656bc |
|
BLAKE2b-256 | 534a9ad8efc38fc15fc3f7e01b6ba063654db4c967ef05b61769bc2bec7d9362 |
Provenance
File details
Details for the file msafilesystem-0.0.8-py3-none-any.whl
.
File metadata
- Download URL: msafilesystem-0.0.8-py3-none-any.whl
- Upload date:
- Size: 9.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.6.1 CPython/3.9.18 Linux/6.2.0-1012-azure
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | bd8eb18eeebf1118aa08189e2e6fecb33d8a8bd3a4f0780a2885ff53664913e3 |
|
MD5 | f00f899a29b7667443822800f5e2d6e4 |
|
BLAKE2b-256 | 2f931055baa97a73e159501058a832b046778cbd84b9968da112b03c94f471a9 |