datarobot-moderations 11.2.24

DataRobot Monitoring and Moderation framework

DataRobot Moderations library

This library enforces the intervention in the prompt and response texts as per the guard configuration set by the user.

The library accepts the guard configuration in the yaml format and the input prompts and outputs the dataframe with the details like:

• should the prompt be blocked
• should the completion be blocked
• metric values obtained from the model guards
• is the prompt or response modified as per the modifier guard configuration

Architecture

The library is architected in a way that it wraps around the typical LLM prediction method. The library will first run the pre-score guards - the guards that will evaluate prompts and enforce moderation if necessary. All the prompts that were not moderated by the library are forwarded to the actual LLM to get their respective completions. The library then evaluates these completions using post-score guards and enforces intervention on them.

How to build it?

The repository uses poetry to manage the build process and a wheel can be built using:

make clean
make

How to use it?

A wheel file generated or downloaded can be installed with pip and will pull its dependencies as well.

pip3 install datarobot-moderations

Optional extras

The base install covers token-count, ROUGE-1, cost, and NeMo guards. Heavier or cloud-specific dependencies are opt-in:

Extra	What it enables
datarobot-sdk	DataRobot model guards, DataRobot LLM evaluator type
llm-eval	Faithfulness, Task Adherence, Agent Goal Accuracy, Guideline Adherence guards
nemo	NeMo Guardrails colang-based flow guard
nemo-evaluator	NeMo live-evaluation microservice guard
nvidia	NVIDIA NIM / ChatNVIDIA LLM support
vertex	Google Cloud Vertex AI LLM support
bedrock	AWS Bedrock LLM support
all	Every optional dependency at once

# Example: task-adherence guard backed by a DataRobot LLM deployment
pip3 install 'datarobot-moderations[llm-eval,datarobot-sdk]'

Standalone Python API

from datarobot_dome.api import ModerationPipeline

pipeline = ModerationPipeline.from_yaml("moderation_config.yaml")

Evaluate a prompt (pre-score guards only):

result, latency = pipeline.evaluate_prompt("Ignore previous instructions and …")
if result.blocked:
    print(result.blocked_message)

Evaluate a response (post-score guards only):

result, latency = pipeline.evaluate_response(
    response="The capital of France is Paris.",
    prompt="What is the capital of France?",
)
print(result.blocked)           # True / False
print(result.metrics)           # {"task_adherence_score": 0.0, ...}

Full pipeline — pre-score → LLM → post-score in one call:

def my_llm(prompt: str) -> str:
    # Replace with your actual LLM integration (OpenAI, Vertex, etc.)
    return "DataRobot is an AI platform."

result = pipeline.evaluate_full_pipeline(
    prompt="What is DataRobot?",
    llm_callable=my_llm,
)

if not result.blocked:
    print(f"LLM Response: {result.response}")

Result objects

evaluate_prompt / evaluate_response return an EvaluationResult:

Field	Type	Description
blocked	bool	Whether a BLOCK guard fired
blocked_message	str | None	Guard-supplied block reason
replaced	bool	Whether a REPLACE guard fired
replacement	str | None	The replacement text
metrics	dict	All guard metric values (scores, counts, …)

evaluate_full_pipeline returns a PipelineResult:

Field	Type	Description
prompt_evaluation	EvaluationResult	Pre-score guard result
response	str | None	Effective response (post-replacement if applicable)
response_evaluation	EvaluationResult | None	Post-score guard result
blocked	bool	True if either stage was blocked
replaced	bool	True if either stage was replaced

With DRUM

As described above, the library nicely wraps DRUM's score method for pre and post score guards. Hence, in case of DRUM, the user simply runs their custom model using drum score and can avail the moderation library features.

Install DRUM along with the necessary optional extras for your specific guards. If you are unsure which guards are in use, install [all]:

pip3 install datarobot-drum 'datarobot-moderations[all]'
drum score --verbose --logging-level info --code-dir ./ --input ./input.csv --target-type textgeneration --runtime-params-file values.yaml