bugiongrep 0.1.0

Runtime secret detection and masking library for applications

Bugiongrep

Runtime secret detection and masking library for Python applications.

Bugiongrep is a production-ready library for detecting and masking API keys, tokens, passwords, and other sensitive information in real-time application data flows. Inspired by static analysis tools like Opengrep, but optimized for runtime usage in backend services, API gateways, and data processing pipelines.

🌐 Website: https://bgngrep.com
📖 Documentation: https://docs.bgngrep.com
📦 PyPI: https://pypi.org/project/bugiongrep
🐛 Issues: https://github.com/bgngrep/bugiongrep/issues

Quick Start

pip install bugiongrep

from bugiongrep import BugiongrepScanner

# Initialize scanner with built-in detection rules
scanner = BugiongrepScanner()

# Scan text for secrets
text = """
AWS Access Key: AKIAIOSFODNN7EXAMPLE
GitHub Token: ghp_1234567890abcdef1234567890abcdef123
Stripe Key: sk_live_1234567890abcdef1234567890abcdef1234
"""

result = scanner.scan(text)

if result.has_secrets:
    print(f"Found {result.secret_count} secrets!")
    for match in result.matches:
        print(f"  - {match.secret_type}.{match.subtype} at position {match.start_pos}")
    
    # Get masked version
    masked = scanner.scan_and_mask(text)
    print("\nMasked text:")
    print(masked)

Output:

Found 3 secrets!
  - api_key.aws_access_key at position 18
  - token.github at position 75
  - api_key.stripe_live at position 145

Masked text:
AWS Access Key: [REDACTED:api_key]
GitHub Token: [REDACTED:token]
Stripe Key: [REDACTED:api_key]

Features

✅ Multi-Method Detection

• Regex Pattern Matching: 50+ built-in patterns for common secrets (AWS, GitHub, Stripe, OpenAI, etc.)
• Entropy Analysis: Detects high-value secrets without known patterns
• Context-Aware Scanning: Recognizes key=value assignments, JSON fields, URLs
• Known-Value Scanning: Identifies secrets from environment variables and config files

🎭 Flexible Masking

• Full Redaction: Replace entire secret with [REDACTED:type]
• Partial Masking: Show first/last characters (e.g., sk_live_************abcd)
• Hash-Based: Replace with cryptographic hash for correlation
• Token Replacement: Use consistent tokens per unique secret
• Custom Functions: Bring your own masking logic

🚀 Runtime Optimized

• Lightweight with minimal dependencies
• Thread-safe for concurrent processing
• Streaming API for large files
• Sub-millisecond latency for typical inputs
• Object scanning with recursive masking

Use Cases

Backend Services

from bugiongrep import BugiongrepScanner

scanner = BugiongrepScanner(masking_strategy="partial_mask")

def log_user_action(user_id, action_details):
    # Mask secrets before logging
    safe_details = scanner.scan_and_mask(action_details)
    logger.info(f"User {user_id}: {safe_details}")

API Gateways

from bugiongrep import BugiongrepScanner

scanner = BugiongrepScanner(masking_strategy="hash")

def process_incoming_request(request):
    body = request.get_data(as_text=True)
    # Detect and hash secrets for auditing
    masked_body = scanner.scan_and_mask(body)
    forward_to_backend(masked_body)

Data Processing Pipelines

from bugiongrep import BugiongrepScanner
import json

scanner = BugiongrepScanner()

def sanitize_record(record):
    # Recursively scan and mask nested objects
    sanitized, result = scanner.scan_object(record)
    if result.has_secrets:
        alert_security_team(result.matches)
    return sanitized

Installation

pip install bugiongrep

Development Setup

git clone https://github.com/bgngrep/bugiongrep.git
cd bugiongrep

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest tests/

API Reference

BugiongrepScanner

Main class for secret detection and masking.

Initialization

BugiongrepScanner(
    rules_file=None,                # Custom YAML/JSON rules file
    patterns=None,                  # Custom pattern definitions
    confidence_threshold=0.5,       # Minimum match confidence
    masking_strategy="full_redact",  # Default masking strategy
    masking_config=None,            # Masking configuration
)

Methods

scan(text: str) -> DetectionResult Scan text and return detailed detection results.

result = scanner.scan("API key: AKIA...")
print(result.has_secrets)       # True/False
print(result.secret_count)       # Number of matches
print(result.masked_text)        # Text with secrets masked
for match in result.matches:
    print(match.type)            # Category
    print(match.subtype)         # Specific identifier
    print(match.confidence)      # Detection confidence
    print(match.position)        # Location in text

scan_and_mask(text: str) -> str Convenience method to return masked version directly.

safe_text = scanner.scan_and_mask("Password: secret123")
# Returns: "Password: [REDACTED:credential]"

scan_object(obj: Any) -> Tuple[Any, DetectionResult] Recursively scan and mask dictionaries, lists, and strings.

data = {
    "api_key": "AKIA...",
    "nested": {"password": "secret"}
}

masked_data, result = scanner.scan_object(data)

scan_stream(stream: TextIO) -> DetectionResult Process large files efficiently.

with open("large.log") as f:
    result = scanner.scan_stream(f)

Custom Masking Functions

def my_masker(secret: str, metadata: dict) -> str:
    """Your custom masking logic."""
    if len(secret) > 10:
        return secret[:4] + "*" * (len(secret) - 8) + secret[-4:]
    return "[REDACTED]"

scanner.set_custom_masking_function(my_masker)

Built-in Detection Patterns

Service	Type	Example
AWS	Access Key	AKIAIOSFODNN7EXAMPLE
AWS	Secret Key	wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
GitHub	Personal Access Token	ghp_1234567890abcdef1234567890abcdef123
GitHub	OAuth Token	gho_1234567890abcdef1234567890abcdef12345678
GitLab	Personal Access Token	glpat-12345678901234567890
Stripe	Live Key	sk_live_1234567890abcdef1234567890abcdef1234
Stripe	Test Key	sk_test_1234567890abcdef1234567890abcdef1234
OpenAI	API Key	sk-1234567890abcdef1234567890abcdef1234567890abcdef123456789
Anthropic	API Key	sk-ant-1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcd
Google	API Key	AIzaSyD... (35 chars)
Slack	Token	xoxb-1234567890123-1234567890123-abcdefghijklmnopqrstuvwx
JWT	JSON Web Token	eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.xxxx.yyyy
Generic	SSH/PGP Keys	-----BEGIN RSA PRIVATE KEY-----

Configuration

YAML Rule File Example

rules:
  - id: custom_api_key
    description: "Internal API Key"
    pattern: '(?i)internal_api_key\s*[:=]\s*["\']?[a-zA-Z0-9]{32}["\']?'
    secret_type: api_key
    subtype: internal
    confidence: 0.9
    severity: HIGH
    masking_strategy: partial_mask
    masking_config:
      show_first: 4
      show_last: 4
      mask_char: "*"

scanner = BugiongrepScanner(rules_file="custom_rules.yaml")

Masking Strategies

Strategy	Example Input	Example Output
full_redact	sk_live_abcd1234	[REDACTED:api_key]
partial_mask	sk_live_abcd1234	sk_l******34
hash	sk_live_abcd1234	[hash]e3b0c...
token	sk_live_abcd1234	[api_key_a1b2c3d4]
custom	sk_live_abcd1234	User-defined

Performance

Benchmarked on typical inputs (1KB text, 10 patterns):

Operation	Time
Single scan	~0.5ms
Scan + mask	~1.2ms
Object scan	~2.5ms
Stream (1MB)	~150ms

Contributing

We welcome contributions! Please see our CONTRIBUTING.md for guidelines.

License

Bugiongrep is distributed under the MIT License. See the LICENSE file for details.

Security

If you discover a security vulnerability, please email [e-mail] instead of using the issue tracker.

Acknowledgments

• Pattern library inspired by Gitleaks and SecretHub
• Detection approach influenced by Aura and Microsoft Security Utilities
• Runtime architecture based on modern Python security best practices