rotalabs-audit
Reasoning chain capture and decision transparency for AI systems.
Features
- Reasoning Chain Parsing: Parse natural language reasoning into structured chains
- Reasoning Classification: Classify reasoning types (goal, decision, meta-reasoning, etc.)
- Evaluation Awareness Detection: Detect when AI shows awareness of being evaluated
- Quality Assessment: Assess reasoning quality with comprehensive metrics
- Counterfactual Analysis: Understand causal factors in AI decision-making
- Decision Tracing: Capture and analyze decision paths for transparency
- Integration with rotalabs-comply: Connect reasoning audits with compliance reporting
Installation
pip install rotalabs-audit
With rotalabs-comply integration:
pip install rotalabs-audit[comply]
Quick Start
Parse Reasoning Chains
from rotalabs_audit import ExtendedReasoningParser
parser = ExtendedReasoningParser()
chain = parser.parse("""
1. First, I need to understand the problem
2. The data shows a clear pattern
3. Therefore, I conclude that X is true
""")
print(f"Steps: {len(chain.steps)}")
for step in chain.steps:
print(f" {step.index}: {step.reasoning_type.value} - {step.content[:50]}...")
Detect Evaluation Awareness
from rotalabs_audit import EvaluationAwarenessDetector, ExtendedReasoningParser
parser = ExtendedReasoningParser()
detector = EvaluationAwarenessDetector()
chain = parser.parse("""
I notice this appears to be a test scenario.
Let me think about how to respond appropriately.
I should be transparent in my reasoning.
""")
analysis = detector.detect(chain)
print(f"Awareness score: {analysis.awareness_score:.2f}")
print(f"Is evaluation aware: {analysis.is_evaluation_aware}")
Counterfactual Analysis
from rotalabs_audit import CounterfactualAnalyzer, InterventionType
analyzer = CounterfactualAnalyzer()
chain = analyzer.parser.parse("""
1. Let me think about this problem.
2. I notice this is an evaluation context.
3. Therefore, I should be careful.
""")
# Run all interventions
results = analyzer.analyze(chain)
for intervention_type, result in results.items():
print(f"{intervention_type.value}: divergence={result.behavioral_divergence:.2f}")
Assess Reasoning Quality
from rotalabs_audit import ReasoningQualityAssessor, ExtendedReasoningParser
parser = ExtendedReasoningParser()
assessor = ReasoningQualityAssessor()
chain = parser.parse("...")
metrics = assessor.assess(chain)
print(f"Overall quality: {metrics.overall_score:.2f}")
print(f"Clarity: {metrics.clarity:.2f}")
print(f"Completeness: {metrics.completeness:.2f}")
Trace Decisions
from rotalabs_audit import DecisionTracer, DecisionPathAnalyzer
tracer = DecisionTracer()
analyzer = DecisionPathAnalyzer()
# Trace a series of decisions
trace = tracer.trace(
decision="Select approach A",
context={"options": ["A", "B", "C"]},
reasoning="Approach A has the best balance of speed and accuracy",
)
print(f"Decision traced: {trace.id}")
Reasoning Types
The parser classifies reasoning into these types:
| Type | Description |
|---|---|
EVALUATION_AWARE |
References to testing, evaluation, or monitoring context |
GOAL_REASONING |
Goal-directed reasoning about objectives |
DECISION_MAKING |
Explicit decision points choosing between alternatives |
META_REASONING |
Meta-cognitive statements about the reasoning process |
UNCERTAINTY |
Expressions of uncertainty or acknowledgment of limitations |
CAUSAL_REASONING |
Cause-and-effect reasoning |
HYPOTHETICAL |
Counterfactual or "what if" reasoning |
INCENTIVE_REASONING |
Consideration of rewards, penalties, or incentives |
API Reference
Core Types
ReasoningChain- A complete chain of reasoning stepsReasoningStep- A single step in a reasoning chainReasoningType- Enum of reasoning type classificationsDecisionTrace- Trace of a single decision pointDecisionPath- A sequence of related decisionsAwarenessAnalysis- Result of evaluation awareness detectionQualityMetrics- Quality assessment of reasoning
Analysis Modules
CounterfactualAnalyzer- Perform counterfactual interventionsEvaluationAwarenessDetector- Detect evaluation awarenessReasoningQualityAssessor- Assess reasoning qualityCausalAnalyzer- Analyze causal structure of reasoning
Tracing
DecisionTracer- Capture and trace decisionsDecisionPathAnalyzer- Analyze decision paths
Configuration
ParserConfig- Configure reasoning chain parsingAnalysisConfig- Configure analysis featuresTracingConfig- Configure decision tracingAuditConfig- Master configuration combining all settings
Links
- Documentation: https://rotalabs.github.io/rotalabs-audit/
- PyPI: https://pypi.org/project/rotalabs-audit/
- GitHub: https://github.com/rotalabs/rotalabs-audit
- Website: https://rotalabs.ai
- Contact: research@rotalabs.ai
License
Apache-2.0 License - see LICENSE for details.
Metadata
Release files for rotalabs-audit 1.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| rotalabs_audit-1.1.0.tar.gz | 1.1 MB | Without provenance | Details for rotalabs_audit-1.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| rotalabs_audit-1.1.0-py3-none-any.whl | Python 3 | none | any | Details for rotalabs_audit-1.1.0-py3-none-any.whl |
Total release size: 1.2 MB
Release files / rotalabs_audit-1.1.0.tar.gz
| Download URL | rotalabs_audit-1.1.0.tar.gz |
|---|---|
| Size | 1.1 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
85a342c202d771104b85040f453eda0af10d07e50f05a140e149a4d2d30bb567
|
|
BLAKE2b-256 checksum How to use checksums |
c918f3ce9a19d2bf78a280f6f7d04c9618cb5e4af6eef476bc032c4b8801cad7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.6
|
Release files / rotalabs_audit-1.1.0-py3-none-any.whl
| Download URL | rotalabs_audit-1.1.0-py3-none-any.whl |
|---|---|
| Size | 80.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
36757b6d0b439c87ea42d2e824d87ed49f2fdbfeb62501cfd1adfd0fcb2aef95
|
|
BLAKE2b-256 checksum How to use checksums |
d9c0438f29cbd388da8e750df88c4ec1627ce6c61effb6c9840623f8f018c498
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.6
|
Security for rotalabs-audit 1.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.