cohere-melody 0.8.0

Templating rendering and generation parsing for Cohere models

melody

[!WARNING] This library is currently in development and the interfaces are subject to change. Be sure to pin this dependency to a specific version.

Templating rendering and generation parsing for Cohere models.

Dev Setup

1. Install Rust from rustup.rs.
2. Install rust-analyzer for IDE support.
3. make rust-test && make rust-lint to ensure everything is working.

Examples

The examples/ directory contains several example programs demonstrating how to use the Melody parsing library in Rust. You can run them using:

cargo run --example basic

Usage

Parsing

use cohere_melody::parsing::Filter;
use cohere_melody::*;

// Create a filter with options
let options = parsing::FilterOptions::new().cmd3();

let mut filter = parsing::new_filter(options);

// Simulate text
let citation_text = "Hello <co: 1>world</co: 1>!";

// Write text — returns an aggregated result
let result = filter.write_decoded(citation_text);

if let Some(text) = &result.content {
    println!("  Content: {text}");
}
for citation in &result.citations {
    println!(
        "    Citation: {} (indices {}-{})",
        citation.text, citation.start_index, citation.end_index
    );
}

// Flush remaining tokens
let final_result = filter.flush_partials();

Templating

use cohere_melody::parsing::Filter;
use cohere_melody::*;

// Assemble inputs into the template (e.g. conversation history)
let options = templating::RenderCmd4Options {
    messages: vec![
        templating::types::Message {
            role: templating::types::Role::System,
            content: vec![templating::types::Content {
                content_type: templating::types::ContentType::Text,
                text: Some("You are a helpful assistant.".to_string()),
                thinking: None,
                image: None,
                document: None,
            }],
            tool_calls: vec![],
            tool_call_id: None,
            citations: vec![],
        },
        templating::types::Message {
            role: templating::types::Role::User,
            content: vec![templating::types::Content {
                content_type: templating::types::ContentType::Text,
                text: Some("Hello Command!.".to_string()),
                thinking: None,
                image: None,
                document: None,
            }],
            tool_calls: vec![],
            tool_call_id: None,
            citations: vec![],
        },
    ],
    ..Default::default()
};

// Render prompt
let prompt = templating::render_cmd4(&options).unwrap();

Releasing

Bump the version in Cargo.toml and push to main. The CI will automatically create a GitHub release, build wheels for all platforms, and publish to PyPI.

Building Python Bindings

Prerequisites (from pyo3)

1. Recommended install uv
2. From root directory, run:

   make python-bindings
   

3. Test the bindings:

   cd rust
   uv run python -c "import cohere_melody;"
   

Debugging

You may run into issues calling the Rust static library from other languages (e.g. Golang via CGO). One effective way to debug these issues is to:

1. Build the library in debug mode. You can do this by adding these lines to the Cargo.toml file:

[profile.release]
debug = true

and then building the library normally: make rust-build-with-tokenizers.

2. Create a binary to debug. In Golang, I create a binary from a test:

go test -count=1 ./gobindings/... -c

3. Use gdb (brew install gdb on macOS).

gdb ./gobindings.test
break melody_render_cmd3
run

4. When it hits the breakpoint, you can step through the Rust code to see where things might be going wrong.