sinapsis-dfine 0.1.4

Training and inference templates based on the D-FINE architecture.

Sinapsis D-FINE

Templates for training and inference with the D-FINE model

🐍 Installation • 🚀 Features • 📚 Usage example • 🌐 Webapp • 📙 Documentation • 🔍 License

The Sinapsis D-FINE module provides templates for training and inference with the D-FINE model, enabling advanced object detection tasks.

🐍 Installation

Install using your package manager of choice. We encourage the use of uv

Example with uv:

  uv pip install sinapsis-dfine --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-dfine --extra-index-url https://pypi.sinapsis.tech

🚀 Features

Templates Supported

The Sinapsis D-FINE module provides two main templates for inference and training:

• DFINETraining: This module implements the training pipeline for the D-FINE model. It includes logic for initializing configuration, downloading weights and setting up the training solver.
• DFINEInference: Template designed to perform inference on a set of images using the different D-FINE architectures available.

🌍 General Attributes

Both templates share the following attributes:

• config_file (str, required): Path to the model configuration file. Refer to the original repo for detailed instructions on using, creating and customizing these configuration files.
• pretrained_model (dict | None, optional): Specifies the size and variant of the pretrained model.
• device (Literal["cpu", "cuda"], required): Defines whether to run inference on CPU or CUDA.
• weights_path (str | None, optional): Path to a custom weights file, if provided. Defaults to None.
• output_dir (str, optional): Directory where downloaded weights will be stored. Defaults to SINAPSIS_CACHE_DIR.

Specific Attributes

There are some attributes specific to the templates used:

• DFINEInference has four additional attributes:
  • threshold (float, required): Confidence score threshold for filtering detections.
  • batch_inference (bool, optional): Whether to perform batch inference. Defaults to False.
  • warmup_iterations (int, optional): Number of warm-up iterations to optimize model performance. Defaults to 10.
  • id2label (dict[int, str] | None, optional): Mapping of class indices to label strings. Required if using custom weights. Defaults to None.
• DFINETraining has five additional attributes:
  • training_mode (Literal["scratch", "tune"], required): "scratch" trains the model from scratch, while "tune" is meant to be used to fine-tune the model with provided or downloaded weights.
  • seed (int | None, optional): Random seed for reproducibility. Defaults to None.
  • use_amp (bool, optional): Enables Automatic Mixed Precision (AMP) for improved performance. Defaults to False.
  • print_rank (int, optional): Rank of the process for logging in distributed training. Defaults to 0.
  • print_method (Literal["builtin", "rich"], optional): Defines the logging method while training. Defaults to "builtin".

[!TIP] Use CLI command sinapsis info --example-template-config TEMPLATE_NAME to produce an example Agent config for the Template specified in TEMPLATE_NAME.

For example, for DFINEInference use sinapsis info --example-template-config DFINEInference to produce an example config like:

agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: DFINEInference
  class_name: DFINEInference
  template_input: InputTemplate
  attributes:
    config_file: '/path/to/config.yml'
    pretrained_model: null
    device: 'cuda'
    weights_path: null
    output_dir: '/path/to/sinapsis/cache'
    threshold: 0.5
    warmup_iterations: 10
    id2label: null

📚 Usage example

The following example demonstrates how to use the DFINEInference template for object detection. This setup processes a folder of images, runs inference using the D-FINE model, and saves the results, including detected bounding boxes.

Config

agent:
  name: dfine_inference
  description: "run inferences with D-FINE"

templates:
  - template_name: InputTemplate
    class_name: InputTemplate
    attributes: {}

  - template_name: FolderImageDatasetCV2
    class_name: FolderImageDatasetCV2
    template_input: InputTemplate
    attributes:
      data_dir: datasets/coco

  - template_name: DFINEInference
    class_name: DFINEInference
    template_input: FolderImageDatasetCV2
    attributes:
      threshold: 0.5
      config_file: artifacts/configs/dfine/dfine_hgnetv2_n_coco.yml
      device: cuda
      output_dir: ./artifacts/dfine_hgnetv2_n_coco
      pretrained_model:
        size: n
        variant: coco

  - template_name: BBoxDrawer
    class_name: BBoxDrawer
    template_input: DFINEInference
    attributes:
      overwrite: true
      randomized_color: false

  - template_name: ImageSaver
    class_name: ImageSaver
    template_input: BBoxDrawer
    attributes:
      root_dir: datasets
      save_dir: output
      extension: png

This configuration defines an agent and a sequence of templates to run object detection with D-FINE.

[!IMPORTANT] The FolderImageDatasetCV2, BBoxDrawer and ImageSaver correspond to sinapsis-data-readers, sinapsis-data-visualization and sinapsis-data-writers. If you want to use the example, please make sure you install the packages.

To run the config, use the CLI:

sinapsis run name_of_config.yml

🌐 Webapp

The webapps included in this project demonstrate the modularity of the templates, showcasing the capabilities of various object detection models for different tasks.

[!IMPORTANT] To run the app, you first need to clone this repository:

git clone [e-mail]:Sinapsis-ai/sinapsis-object-detection.git
cd sinapsis-object-detection

[!NOTE] If you'd like to enable external app sharing in Gradio, export GRADIO_SHARE_APP=True

[!NOTE] Agent configuration can be changed through the AGENT_CONFIG_PATH env var. You can check the available configurations in each package configs folder.

[!NOTE] When running the app with the D-FINE model, it defaults to a confidence threshold of 0.5, uses CUDA for acceleration, and employs the nano-sized D-FINE model trained on the COCO dataset. These settings can be customized by modifying the demo.yml file inside the configs directory of the sinapsis-dfine package and restarting the webapp.

🐳 Docker

IMPORTANT This docker image depends on the sinapsis-nvidia:base image. Please refer to the official sinapsis instructions to Build with Docker.

1. Build the sinapsis-object-detection image:

docker compose -f docker/compose.yaml build

2. Start the app container:

docker compose -f docker/compose_apps.yaml up sinapsis-dfine-gradio -d

3. Check the status:

docker logs -f sinapsis-dfine-gradio

4. The logs will display the URL to access the webapp, e.g.:

Running on local URL:  http://127.0.0.1:7860

NOTE: The url can be different, check the output of logs.

5. To stop the app:

docker compose -f docker/compose_apps.yaml down

💻 UV

To run the webapp using the uv package manager, follow these steps:

1. Create the virtual environment and sync the dependencies:

uv sync --frozen

2. Install the sinapsis-object-detection package:

uv pip install sinapsis-object-detection[all] --extra-index-url https://pypi.sinapsis.tech

3. Run the webapp:

uv run webapps/detection_demo.py

4. The terminal will display the URL to access the webapp, e.g.:

Running on local URL:  http://127.0.0.1:7860

NOTE: The url can be different, check the output of the terminal.

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.