sinapsis-data-tools 0.2.6

Module that contains different packages to perform data-related operations through Sinapsis templates.

Sinapsis Data Tools

Mono repo with packages to read, write, process data, including images, audios, videos, bytes objects. The packages can be easily extensible to handle other types of data.

🐍 Installation • 📦 Packages • 📚 Usage example • 📙 Documentation • 🔍 License

🐍 Installation

This mono repo consists of different packages to handle data:

• sinapsis-data-analysis
• sinapsis-data-readers
• sinapsis-data-visualization
• sinapsis-data-writers
• sinapsis-generic-data-tools

Install using your package manager of choice. We encourage the use of uv

Example with uv:

  uv pip install sinapsis-data-readers --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-data-readers --extra-index-url https://pypi.sinapsis.tech

Change the name of the package for the one you want to install.

[!IMPORTANT] Templates in each package may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

with uv:

  uv pip install sinapsis-data-readers[all] --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-data-readers[all] --extra-index-url https://pypi.sinapsis.tech

Change the name of the package accordingly.

[!TIP] You can also install all the packages within this project:

  uv pip install sinapsis-data-tools[all] --extra-index-url https://pypi.sinapsis.tech

[!NOTE] Some templates also need system dependencies (e.g., ffmpeg). The installation depends on your OS. For Linux:

apt-get install -y ffmpeg

📦 Packages

Packages summary

• Sinapsis Data Readers

  • Audio Readers
    Read audio files from several formats using Pydub, Soundfile, among others.
  • Dataset Readers
    Read and manipulate tabular datasets from the scikit libraries, among others.
  • Image Readers
    Read and manipulate images from COCO, paths in CSVs, whole folders, etc.
  • Text Readers
    Read text data from a simple string and other sources.
  • Video Readers
    Read videoframes using CV2, Dali, FFMPEG, Torch, among others.

• Sinapsis Data Visualization
  Visualize data distributions and manifolds, as well as draw all kinds of annotations on images, such as bounding boxes, keypoints, labels, oriented bounding boxes, segmentation masks, etc.

• Sinapsis Data Writers
  Write data to many kinds of files.

  • Annotation Writers
    Save text annotations to JSON, geometries to polygons, etc.
  • Audio Writers
    Save to audio files using Soundfile, among others.
  • Image Writers Save to image files using CV2, among others.
  • Video Writers
    Save to video files using CV2 or FFMPEG, among others.

• Sinapsis Generic Data Tools
  Wide range of miscellaneous tools to manipulate your data.

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis Data Tools.

[!TIP] Use CLI command sinapsis info --example-template-config TEMPLATE_NAME to produce an example Agent config for the Template specified in TEMPLATE_NAME.

[!TIP] Run the docker image docker run -it --gpus all sinapsis-data-tools:base bash You need to activate the environment inside the image source .venv/bin/activate

For example, for ImageSaver use sinapsis info --example-template-config ImageSaver to produce the following example config:

agent:
  name: my_test_agent
  description: agent to save image locally
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: ImageSaver
  class_name: ImageSaver
  template_input: InputTemplate
  attributes:
    save_dir: /path/to/save/dir
    extension: jpg
    root_dir: '/path/to/sinapsis/cache'
    save_full_image: true
    save_bbox_crops: false
    save_mask_crops: false
    min_bbox_dim: 5

📚 Usage example

Example agent config

You can copy and paste the following config and run it using the sinapsis cli, changing the data_dir attribute in the FolderImageDatasetCV2 and the root_dir attribute in the ImageSaver template

agent:
  name: my_test_agent
  description: agent to save image locally
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: FolderImageDatasetCV2
  class_name: FolderImageDatasetCV2
  attributes:
    data_dir: /path/to/image
    pattern: '**/*'
    batch_size: 1
    load_on_init: true
    label_path_index: 0
    is_ground_truth: false

- template_name: ImageSaver
  class_name: ImageSaver
  template_input: FolderImageDatasetCV2
  attributes:
    save_dir: /path/to/save/dir
    extension: jpg
    root_dir: '/path/to/sinapsis/cache'
    save_full_image: true
    save_bbox_crops: false
    save_mask_crops: false
    min_bbox_dim: 5

To run, simply use:

sinapsis run name_of_the_config.yml

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.