sdg-feature-combos 0.1.0

Compute feature combinations for LLM-based AttrPrompt synthetic text data generation.

sdg-feature-combos

A Python package for structured synthetic text generation using attributed prompts. Features are named attributes (e.g. topic, label, length) that control variation in LLM-generated text; prompts are Jinja2 templates with {{feature_name}}_{{feature_type}} placeholders that are dynamically populated.

This package:

• Defines a features catalog: a YAML schema for configuring features, their specifications and sampling frequencies.
• Defines a {{feature_name}}_{{feature_type}} placeholder convention that prompt templates must follow.
• Provides build_feature_combos(): a function that computes every valid combination of feature values and allocates an integer count to each, proportional to its joint frequency.
• Provides format_template_vars(): a function that converts each feature combination into Jinja2-ready template variables, so rendered prompts can be passed directly to the LLM.

---

Use Case

This package implements the attributed prompts methodology for LLM-based synthetic text generation, as described in:

AttrPrompt in Action: Evaluating Synthetic Data Generation for SDG Classification (Bolz et al., SwissText 2024)

The core idea is to condition LLM prompts on combinations of named attributes (features), such as topic or length, to produce diverse synthetic text.

---

What is a Features Catalog?

A features catalog is a YAML file where the top-level keys are dataset names, each containing named features organised into two types:

• rule: a standalone LLM directive. For example, a safety feature might take values: "Generate a safe observation." or "Generate an unsafe observation.". The rule feature value is the instruction.

• var: stands for 'variable', a value inserted into an existing instruction. For example, a length feature might have values "10-20" or "50-100" that get substituted into "Write an observation of {{length_var}} words.". The var feature value is a variable in an instruction.

Each feature has a name and a set of values it can take. Each value is represented by three fields:

• value: the text passed to the LLM (the directive or variable value)
• flag: a short identifier used to mark the corresponding output column (e.g. a safe column with value 1 or 0)
• frequency: non-negative sampling weight; must either all be 0 (to exclude the feature) or sum to exactly 1.0 across all values for that feature

Prompt template

Template placeholders follow the {{feature_name}}_{{feature_type}} pattern:

Write a process safety observation.

Follow these rules:
## {{safe_rule}} 
## {{topic_rule}}
## Your observation must be between {{length_var}} words.

---

Features Catalog YAML Structure

<dataset_name>:
  rule:
    - <feature_name>:
      - flag: <identifier>
        value: "<complete LLM directive>"
        frequency: <float>   # must be non-negative & sum to 1.0 across all values
      - flag: <identifier>
        value: "<complete LLM directive>"
        frequency: <float>
  var:
    - <feature_name>:
      - flag: <identifier>
        value: "<value to insert into template>"
        frequency: <float>   # must be non-negative & sum to 1.0 across all values

Multiple datasets can live in the same YAML features catalog. Each dataset can have any number of rule and/or var features, and each feature can have any number of values. Dataset names, feature names, and feature value names must all be unique.

Example

process_safety_obs:
  rule:
    - safe:
      - flag: 1
        value: "Generate a safe observation."
        frequency: 0.35
      - flag: 0
        value: "Generate an unsafe observation."
        frequency: 0.65
    - topic:
      - flag: pdo_uncontroll_motion
        value: "Your observation should relate to Potential Dropped Objects (PDO)."
        frequency: 0.25
      - flag: barriers
        value: "Your observation should relate to the use of barriers."
        frequency: 0.25
      - flag: human_factors
        value: "Your observation must center around the human factors."
        frequency: 0.50
  var:
    - length:   # refers to length of text to generate
      - flag: very_short
        value: "5-15"
        frequency: 0.28
      - flag: short
        value: "15-40"
        frequency: 0.29
      - flag: medium
        value: "35-50"
        frequency: 0.28
      - flag: long
        value: "50-80"
        frequency: 0.15

For the example above, this package would compute 2 × 3 × 4 = 24 feature combinations (all non-zero frequency permutations of safe × topic × length) and allocate counts to each proportionally to their joint frequency.

---

Installation

uv add sdg-feature-combos

Or with pip:

pip install sdg-feature-combos

Requires Python 3.10 or later.

---

Usage

from sdg_feature_combos import build_feature_combos, format_template_vars
from jinja2 import Template

combos = build_feature_combos(
    yaml_path="examples/features_catalog.yaml",
    dataset_nm="process_safety_obs",
    total_num_to_generate=1000,
)

template = Template(open("examples/prompt.txt").read())

results = []
for combo in combos:
    prompt = template.render(
        **format_template_vars(combo["features_by_type"])
    )
    for _ in range(combo["count"]):
        output = call_llm(prompt)  # your LLM call here
        results.append({**combo["features_name_flag_map"], "text": output})
        # each result row: {"safe": 1, "topic": "pdo_uncontroll_motion", "length": "very_short", "text": "..."}

Output structure

build_feature_combos returns a list[FeatureCombo] — one per valid feature combination. Here is what a single FeatureCombo looks like:

{
    "count": 49,
    "features_by_type": {
        "rule": [
            {"safe": "Generate a safe observation."},
            {"topic": "Your observation should relate to Potential Dropped Objects (PDO)."},
        ],
        "var": [{"length": "5-15"}],
    },
    "features_name_flag_map": {
        "safe": 1,
        "topic": "pdo_uncontroll_motion",
        "length": "very_short",
    },
}

---

API

build_feature_combos(yaml_path, dataset_nm, total_num_to_generate)

Load the YAML features catalog at yaml_path, extract the named dataset, and return stratified feature combination counts.

Raises KeyError if dataset_nm is not found in the features catalog.
Raises ValueError if the YAML contains duplicate keys; or if the dataset contains unknown feature types, duplicate feature names, feature values with missing or unexpected keys, or invalid frequencies (non-numeric, negative, or not summing to 1.0).

format_template_vars(features_by_type)

Convert features_by_type into a flat dict[str, str] of Jinja2 template variables, keyed by {feature_name}_{feature_type}.

load_catalog(yaml_path)

Load and return the raw features catalog dict from a YAML file.

Raises ValueError if the YAML contains any duplicate keys.

---

Contributing

Bug reports and pull requests are welcome on GitHub. Please open an issue before submitting a pull request for significant changes.

License

MIT License. See LICENSE for details.