File System IO

Badges
License
PyPI

Version	Build Status
Main

File System IO or simply FSIO is a Python package containing useful file system operations.

Table of contents

• Installation
• Getting Started
• CLI
• Development

Installation

Installing the main package is as simple as downloading from PyPI.

pip install 'fsio==$VERSION'

If you also require use of the CLI, then you can install with extras as follows.

pip install 'fsio[cli]==$VERSION'

Getting Started

This package was designed for simplifying some file system operations. Its original design was for detecting file types of a file, but it has been expanded beyond this. The main code lives within the core directory. Here, for example, you will see the file_type class which supports the detect-file-type commands within the CLI. Moreover, we can use this within Python code provided we have the object in BytesIO form. For example, suppose we have a .parquet file without an extension and we want to establish its type, and confirm that it really is of type parquet. To do this, we could do something as follows.

>>> from io import BytesIO
>>> from pathlib import Path
>>>
>>> from fsio.core import FileType
>>>
>>> path_to_file = Path('path/to/suspected/parquet')
>>>
>>> with path_to_file.open('rb') as f:
>>>     body = BytesIO(f.read())
>>>
>>> FileType.detect_file_type(body)
'parquet'

CLI

If you optionally installed the cli subpackage, then you get extra functionality and are able to use most of the functionality from the core package. For example, you can detect the file type of a given file using the command

fsio detect-file-type path/to/file.ext

This will return the type of the file to the stdout, for example parquet. You can get more information about each command by using the --help flag on each command. The CLI was designed using typer, from the creators of FastAPI. A list of the following CLI commands which are available are below.

Development

To contribute to this project, you can clone the repository and install the development dependencies. This can be done using uv sync --all-extras (which uses the uv.lock file for consistency). There is a pre-commit hook which will run on each commit to ensure that the code is formatted correctly. To install this, we need to run

uv sync --extra test

to install pre-commit and then run pre-commit install in terminal. We can manually run the pre-commit hooks using the command

pre-commit run --all-files

Branching

When creating a new branch, please use one of the following:

• feature/* for new features (branch from and PR to develop);
• bugfix/* for bug fixes (non-critical) (branch from and PR to develop);
• hotfix/* for hotfixes (branch from and PR to main, syncing to develop); and
• release/* for releases (branch from develop with PR to main, syncing to develop).

The main branch is the stable deployment, and release branches are used for preparing a new release.