Skip to main content

Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc Optimized for use with FastAPI/Pydantic.

Project description

msaFilesystem Logo


msaFilesystem - Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc
Optimized for use with FastAPI/Pydantic.
Package version Supported Python versions


Alpha/EarlyStage !!!

Documentation: msaFilesystem Documentation (https://msaFilesystem.u2d.ai/)


Features

  • Abstract Filesystem: Agnostic Abstract Filesystem API which allows to use S3, GCS, Azure Datalake, your local FS, Youtube etc. Based on: PyFilesystem2
  • App Filesystems: Manage filesystems in platform-specific application directories. These classes abstract away the different requirements for user data across platforms.
  • FTP Filesystem: A FTP (File Transport Protocol) Filesystem. Optionally, the connection can be made securely via TLS. This is known as FTPS, or FTP Secure.
  • Memory Filesystem: A filesystem that stored in memory. Memory filesystems are useful for caches, temporary data stores, unit testing, etc.
  • Mount Filesystem: A Mount FS is a virtual filesystem which can seamlessly map sub-directories on to other filesystems.
  • Multi Filesystem: A MultiFS is a filesystem composed of a sequence of other filesystems, where the directory structure of each overlays the previous filesystem in the sequence.
  • OS Filesystem: Manage the filesystem provided by your OS. In essence, an OSFS is a thin layer over the io and os modules of the Python standard library.
  • Sub Filesystem: Manage a directory in a parent filesystem. A SubFS is a filesystem object that maps to a sub-directory of another filesystem.
  • Tar Filesystem: Read and write tar files.
  • Temporary Filesystem: Manage filesystems in temporary locations. A temporary filesytem is stored in a location defined by your OS (/tmp on linux). The contents are deleted when the filesystem is closed.
  • Zip Filesystem: Read and write zip files.
  • SMB Filesystem: A filesystem over SMB.
  • WebDAV Filesystem: A filesystem for WebDAV.
  • Azure Datalake & S3FS Filesystem: A filesystem for Azure Datalake storage & S3FS.
  • Google Cloud Storage (GCS) Filesystem: A filesystem for Google Cloud Storage (GCS). With GCSFS, you can interact with Google Cloud Storage as if it was a regular filesystem.
  • Google Drive Filesystem: A filesystem for Google Drive. Interact with Google Drive as if it was a regular filesystem.
  • Dropbox Filesystem: A filesystem for Dropbox.
  • OneDrive Filesystem: A filesystem for Dropbox.
  • YouTube Videos and Playlists Filesystem: A filesystem for YouTube Videos and Playlists.
  • External Filesystems: See the following wiki page for a list of filesystems not in the core library, and community contributed filesystems.

Main Dependencies

  • fs~=2.4.16: Module that provides a common interface to any filesystem
  • six~=1.16.0: Python 2 and 3 compatibility utilities
  • pysmb~=1.2.8: SMB/CIFS library
  • fs.webdavfs~=0.4.2: WebDAV support
  • fs-dlk~=0.1.3: Azure Datalake support
  • fs-s3fs~=1.1.1: Amazon S3 filesystem support
  • fs-gcsfs~=1.5.1: Google Cloud Storage (GCS) support
  • fs.googledrivefs~=2.3.0: Google Drive support
  • fs.dropboxfs~=0.2.2.post2: Dropbox support
  • fs.onedrivefs~=1.1.1: OneDrive support
  • fs.youtube~=0.3.1: Youtube support
  • fs.smbfs~=1.0.5: SMB support

License Agreement

  • msaFilesystem is based on MIT open source and free to use, it is free for commercial use, but please show/list the copyright information about msaFilesystem somewhere.

How to create the documentation

We use mkdocs and mkdocsstring. The code reference and nav entry get's created virtually by the triggered python script /docs/gen_ref_pages.py while mkdocs serve or build is executed.

Requirements Install for the PDF creation option:

PDF Export is using mainly weasyprint, if you get some errors here pls. check there documentation. Installation is part of the msaFilesystem, so this should be fine.

We can now test and view our documentation using:

mkdocs serve

Build static Site:

mkdocs build

Build and Publish

Build:

python setup.py sdist

Publish to pypi:

twine upload dist/*

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

msafilesystem-0.0.8.tar.gz (8.2 kB view details)

Uploaded Source

Built Distribution

msafilesystem-0.0.8-py3-none-any.whl (9.1 kB view details)

Uploaded Python 3

File details

Details for the file msafilesystem-0.0.8.tar.gz.

File metadata

  • Download URL: msafilesystem-0.0.8.tar.gz
  • Upload date:
  • Size: 8.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.6.1 CPython/3.9.18 Linux/6.2.0-1012-azure

File hashes

Hashes for msafilesystem-0.0.8.tar.gz
Algorithm Hash digest
SHA256 7d44f0556a718a537ce0d5275f859e94b9e5e2c7d121c3aaf62e248a8d138f28
MD5 f06046e0e605e206bbd609c4feb656bc
BLAKE2b-256 534a9ad8efc38fc15fc3f7e01b6ba063654db4c967ef05b61769bc2bec7d9362

See more details on using hashes here.

File details

Details for the file msafilesystem-0.0.8-py3-none-any.whl.

File metadata

  • Download URL: msafilesystem-0.0.8-py3-none-any.whl
  • Upload date:
  • Size: 9.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.6.1 CPython/3.9.18 Linux/6.2.0-1012-azure

File hashes

Hashes for msafilesystem-0.0.8-py3-none-any.whl
Algorithm Hash digest
SHA256 bd8eb18eeebf1118aa08189e2e6fecb33d8a8bd3a4f0780a2885ff53664913e3
MD5 f00f899a29b7667443822800f5e2d6e4
BLAKE2b-256 2f931055baa97a73e159501058a832b046778cbd84b9968da112b03c94f471a9

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page