Sinapsis Vowpal-Wabbit

High-performance Contextual Bandits and Reinforcement Learning using Vowpal Wabbit

🐍 Installation • 🚀 Features • 📚 Usage example • 📙 Documentation • 🔍 License

Sinapsis Vowpal Wabbit provides a modular framework for implementing real-time decision-making systems. By leveraging the industry-standard Vowpal Wabbit engine, this module enables efficient online learning and reinforcement learning (RL) workflows with minimal latency.

🐍 Installation

Install using your package manager of choice. We encourage the use of uv

Example with uv:

  uv pip install sinapsis-vowpal-wabbit --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-vowpal-wabbit --extra-index-url https://pypi.sinapsis.tech

[!IMPORTANT] Templates may require extra dependencies. For development, we recommend installing the package with all the optional dependencies:

with uv:

  uv pip install sinapsis-vowpal-wabbit[all] --extra-index-url https://pypi.sinapsis.tech

or with raw pip:

  pip install sinapsis-vowpal-wabbit[all] --extra-index-url https://pypi.sinapsis.tech

[!TIP] Use CLI command sinapsis info --all-template-names to show a list with all the available Template names installed with Sinapsis OCR.

[!TIP] Use CLI command sinapsis info --example-template-config CBExploreADFPredict to produce an example Agent config for the CBExploreADFPredict template.

🚀 Features

Templates Supported

This repository implements specific templates for Vowpal Wabbit's most powerful reductions:

• CBExploreADFPredict: Handles real-time inference for Contextual Bandits using Action-Dependent Features (ADF).

• CBExploreADFLearn: Template to perform offline learning. It iterates through historical datasets (context, action, reward) for a defined number of epochs to optimize the model weights before deployment.

• CBExploreADFPredictEmbeddings: Handles real-time inference for Contextual Bandits using Action-Dependent Features (ADF) enhanced with text embeddings. This template processes incoming context and actions by combining traditional raw text features with vector-based embeddings to provide more nuanced, semantically aware predictions in production environments.

• CBExploreADFLearnEmbeddings: Template for offline learning that incorporates text embeddings into the optimization process. It iterates through historical datasets (context, actions, and rewards) over a defined number of epochs. By training on a combination of raw text and sentence based embeddings, it captures deeper semantic relationships to more effectively optimize model weights prior to deployment.

📚 Usage example

agent:
  name: my_test_agent
templates:
- template_name: InputTemplate
  class_name: InputTemplate
  attributes: {}
- template_name: CBExploreADFPredict
  class_name: CBExploreADFPredict
  template_input: InputTemplate
  attributes:
    stop_words: ["from", "at", "i", "you"]
    clean_text_pattern: '[^a-z0-9\s]'
    actions: ["action_1", "action_2","action_3"]
    vw_workspace_params:
      bit_precision: 24
      quadratic_interactions: ["CA"]
      cubic_interactions: []
      ngram_namespaces: ["C"]
      ngram_size: 2
      learning_rate: 0.5
      l1: 0.0
      l2: 0.0
      exploration_method: epsilon
      exploration_value: 0.2
      adaptive: true
      normalized: true
      quiet: false
    remove_stop_words: false
    remove_special_characters: true
    model_path: "artifacts/model.vw"
    inference_only: true
    threshold: 0
    top_k: 3

To run, simply use:

sinapsis run name_of_the_config.yml

📙 Documentation

Documentation for this and other sinapsis packages is available on the sinapsis website

Tutorials for different projects within sinapsis are available at sinapsis tutorials page

🔍 License

This project is licensed under the AGPLv3 license, which encourages open collaboration and sharing. For more details, please refer to the LICENSE file.

For commercial use, please refer to our official Sinapsis website for information on obtaining a commercial license.