Skip to main content

Implement the concept of Storage with amazon S3, sftp...

Project description

Odoo Community Association

Filesystem Storage Backend

Beta License: LGPL-3 OCA/storage Translate me on Weblate Try me on Runboat

This addon is a technical addon that allows you to define filesystem like storage for your data. It’s used by other addons to store their data in a transparent way into different kind of storages.

Through the fs.storage record, you get access to an object that implements the fsspec.spec.AbstractFileSystem interface and therefore give you an unified interface to access your data whatever the storage protocol you decide to use.

The list of supported protocols depends on the installed fsspec implementations. By default, the addon will install the following protocols:

  • LocalFileSystem

  • MemoryFileSystem

  • ZipFileSystem

  • TarFileSystem

  • FTPFileSystem

  • CachingFileSystem

  • WholeFileSystem

  • SimplCacheFileSystem

  • ReferenceFileSystem

  • GenericFileSystem

  • DirFileSystem

  • DatabricksFileSystem

  • GitHubFileSystem

  • JupiterFileSystem

  • OdooFileSystem

The OdooFileSystem is the one that allows you to store your data into a directory mounted into your Odoo’s storage directory. This is the default FS Storage when creating a new fs.storage record.

Others protocols are available through the installation of additional python packages:

  • DropboxDriveFileSystem -> pip install fsspec[dropbox]

  • HTTPFileSystem -> pip install fsspec[http]

  • HTTPSFileSystem -> pip install fsspec[http]

  • GCSFileSystem -> pip install fsspec[gcs]

  • GSFileSystem -> pip install fsspec[gs]

  • GoogleDriveFileSystem -> pip install gdrivefs

  • SFTPFileSystem -> pip install fsspec[sftp]

  • HaddoopFileSystem -> pip install fsspec[hdfs]

  • S3FileSystem -> pip install fsspec[s3]

  • WandbFS -> pip install wandbfs

  • OCIFileSystem -> pip install fsspec[oci]

  • AsyncLocalFileSystem -> pip install ‘morefs[asynclocalfs]

  • AzureDatalakeFileSystem -> pip install fsspec[adl]

  • AzureBlobFileSystem -> pip install fsspec[abfs]

  • DaskWorkerFileSystem -> pip install fsspec[dask]

  • GitFileSystem -> pip install fsspec[git]

  • SMBFileSystem -> pip install fsspec[smb]

  • LibArchiveFileSystem -> pip install fsspec[libarchive]

  • OSSFileSystem -> pip install ossfs

  • WebdavFileSystem -> pip install webdav4

  • DVCFileSystem -> pip install dvc

  • XRootDFileSystem -> pip install fsspec-xrootd

This list of supported protocols is not exhaustive or could change in the future depending on the fsspec releases. You can find more information about the supported protocols on the fsspec documentation.

Table of contents

Usage

Configuration

When you create a new backend, you must specify the following:

  • The name of the backend. This is the name that will be used to identify the backend into Odoo

  • The code of the backend. This code will identify the backend into the store_fname field of the ir.attachment model. This code must be unique. It will be used as scheme. example of the store_fname field: odoofs://abs34Tg11.

  • The protocol used by the backend. The protocol refers to the supported protocols of the fsspec python package.

  • A directory path. This is a root directory from which the filesystem will be mounted. This directory must exist.

  • The protocol options. These are the options that will be passed to the fsspec python package when creating the filesystem. These options depend on the protocol used and are described in the fsspec documentation.

  • Resolve env vars. This options resolves the protocol options values starting with $ from environment variables

  • Check Connection Method. If set, Odoo will always check the connection before using a storage and it will remove the fs connection from the cache if the check fails.

    • Create Marker file : create a hidden file on remote and then check it exists with Use it if you have write access to the remote and if it is not an issue to leave the marker file in the root directory.

    • List file : list all files from the root directory. You can use it if the directory path does not contain a big list of files (for performance reasons)

Some protocols defined in the fsspec package are wrappers around other protocols. For example, the SimpleCacheFileSystem protocol is a wrapper around any local filesystem protocol. In such cases, you must specify into the protocol options the protocol to be wrapped and the options to be passed to the wrapped protocol.

For example, if you want to create a backend that uses the SimpleCacheFileSystem protocol, after selecting the SimpleCacheFileSystem protocol, you must specify the protocol options as follows:

{
    "directory_path": "/tmp/my_backend",
    "target_protocol": "odoofs",
    "target_options": {...},
}

In this example, the SimpleCacheFileSystem protocol will be used as a wrapper around the odoofs protocol.

Known issues / Roadmap

  • Transactions: fsspec comes with a transactional mechanism that once started, gathers all the files created during the transaction, and if the transaction is committed, moves them to their final locations. It would be useful to bridge this with the transactional mechanism of odoo. This would allow to ensure that all the files created during a transaction are either all moved to their final locations, or all deleted if the transaction is rolled back. This mechanism is only valid for files created during the transaction by a call to the open method of the file system. It is not valid for others operations, such as rm, mv_file, … .

Changelog

16.0.1.5.0 (2026-05-26)

Features

  • Replace {db_name} by the database name in directory_path (#db_name)

16.0.1.2.0 (2024-02-06)

Features

  • Invalidate FS filesystem object cache when the connection fails, forcing a reconnection. (#320)

16.0.1.1.0 (2023-12-22)

Features

  • Add parameter on storage backend to resolve protocol options values starting with $ from environment variables (#303)

16.0.1.0.3 (2023-10-17)

Bugfixes

  • Fix access to technical models to be able to upload attachments for users with basic access (#289)

16.0.1.0.2 (2023-10-09)

Bugfixes

  • Avoid config error when using the webdav protocol. The auth option is expected to be a tuple not a list. Since our config is loaded from a json file, we cannot use tuples. The fix converts the list to a tuple when the config is related to a webdav protocol and the auth option is into the confix. (#285)

Bug Tracker

Bugs are tracked on GitHub Issues. In case of trouble, please check there if your issue has already been reported. If you spotted it first, help us to smash it by providing a detailed and welcomed feedback.

Do not contact contributors directly about support or help with technical issues.

Credits

Authors

  • ACSONE SA/NV

Contributors

Maintainers

This module is maintained by the OCA.

Odoo Community Association

OCA, or the Odoo Community Association, is a nonprofit organization whose mission is to support the collaborative development of Odoo features and promote its widespread use.

This module is part of the OCA/storage project on GitHub.

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

odoo_addon_fs_storage-16.0.2.0.0.3-py3-none-any.whl (62.2 kB view details)

Uploaded Python 3

File details

Details for the file odoo_addon_fs_storage-16.0.2.0.0.3-py3-none-any.whl.

File metadata

File hashes

Hashes for odoo_addon_fs_storage-16.0.2.0.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 b56b337b48da258390edc2aa5656d393e0c5e1fefc08cd9ca80adc3e2fce9bf0
MD5 2a0d7d9118b9013fa89e7248d2830185
BLAKE2b-256 6e82b73a26721e6c41b498e26185ed6f82748695ee10fc07aceee0c877c56b17

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page