recator 0.1.2

Automated code duplicate detection and refactoring library

Recator 🔧

Recator - Automated code duplicate detection and refactoring library for Python

📋 Overview

Recator is a powerful Python library that automatically detects and refactors code duplicates across multiple programming languages using simple heuristics without requiring LLMs. It works efficiently on CPU and supports various programming languages including Python, JavaScript, Java, C/C++, and more.

✨ Features

• Multi-language Support: Python, JavaScript, Java, C/C++, C#, PHP, Ruby, Go, Rust, Kotlin, Swift
• Multiple Detection Algorithms:
  • Exact duplicate detection (hash-based)
  • Token-based similarity detection
  • Fuzzy matching using sequence comparison
  • Structural similarity detection (same structure, different names)
• Automated Refactoring Strategies:
  • Extract Method - for duplicate code blocks
  • Extract Class - for structural duplicates
  • Extract Module - for file-level duplicates
  • Parameterize - for similar code with differences
• Safe Mode: Creates .refactored versions without modifying originals
• CPU Efficient: Uses simple heuristics, no GPU or LLM required
• Configurable: Adjustable thresholds and parameters

🚀 Installation

# Basic installation
pip install recator

# Or install from source
git clone https://github.com/pyfunc/recator.git
cd recator
pip install -e .

# Install with development dependencies
pip install -e ".[dev]"

# Install with advanced features
pip install -e ".[advanced]"

📖 Usage

Command Line Interface

# Basic analysis
recator /path/to/project

# Verbose analysis with custom parameters
recator /path/to/project -v --min-lines 6 --threshold 0.9

# Preview refactoring suggestions
recator /path/to/project --refactor

# Show duplicate code snippets (first N) during analysis
recator /path/to/project --analyze --show-snippets --max-show 5 -v

# Interactive selection of duplicates to refactor (dry-run preview)
recator /path/to/project --refactor --interactive --dry-run

# Refactor on demand by selecting duplicates (IDs or ranges)
# Example selects IDs 1, 3, 4, 5
recator /path/to/project --refactor --select 1,3-5 --dry-run

# Apply refactoring (creates .refactored files)
recator /path/to/project --refactor --apply

# Analyze specific languages only
recator /path/to/project --languages python javascript

# Exclude patterns
recator /path/to/project --exclude "*.test.js" "build/*"

# Save results to JSON
recator /path/to/project --output results.json

# Show duplicate code snippets (once per duplicate) and all occurrences
recator /path/to/project --analyze --show-snippets --max-show 0 --max-blocks 0 -v

# Suppress overlapping/near-identical groups (default) vs show all
recator /path/to/project --analyze -v                         # suppressed
recator /path/to/project --analyze -v --no-suppress-duplicates  # no suppression

# Control snippet preview size in verbose mode
recator /path/to/project --analyze -v --snippet-lines 12

Python API

from recator import Recator

# Initialize with project path
recator = Recator('/path/to/project')

# Analyze for duplicates
results = recator.analyze()
print(f"Found {results['duplicates_found']} duplicates")

# Get detailed duplicate information
for duplicate in results['duplicates']:
    print(f"Type: {duplicate['type']}")
    print(f"Files: {duplicate.get('files', [])}")
    print(f"Confidence: {duplicate.get('confidence', 0)}")

# Preview refactoring
preview = recator.refactor_duplicates(dry_run=True)
print(f"Estimated LOC reduction: {preview['estimated_loc_reduction']}")

# Apply refactoring
refactoring_results = recator.refactor_duplicates(dry_run=False)
print(f"Modified {len(refactoring_results['modified_files'])} files")

Custom Configuration

from recator import Recator

config = {
    'min_lines': 5,                    # Minimum lines for duplicate
    'min_tokens': 40,                  # Minimum tokens for duplicate
    'similarity_threshold': 0.90,      # Similarity threshold (0-1)
    'languages': ['python', 'java'],   # Languages to analyze
    'exclude_patterns': ['*.min.js'],  # Patterns to exclude
    'safe_mode': True,                 # Don't modify originals
}

recator = Recator('/path/to/project', config)
results = recator.analyze()

🔍 Detection Algorithms

1. Exact Duplicate Detection

Finds identical code blocks using hash comparison.

2. Token-based Detection

Compares token sequences to find duplicates that may have different formatting.

3. Fuzzy Matching

Uses sequence matching algorithms to find similar (but not identical) code.

4. Structural Detection

Identifies code with the same structure but different variable/function names.

🛠️ Refactoring Strategies

Extract Method

# Before: Duplicate blocks in multiple places
def process_user(user):
    # validation block (duplicate)
    if not user.email:
        raise ValueError("Email required")
    if "@" not in user.email:
        raise ValueError("Invalid email")
    # ... processing

def update_user(user):
    # validation block (duplicate)
    if not user.email:
        raise ValueError("Email required")
    if "@" not in user.email:
        raise ValueError("Invalid email")
    # ... updating

# After: Extracted method
def validate_user_email(user):
    if not user.email:
        raise ValueError("Email required")
    if "@" not in user.email:
        raise ValueError("Invalid email")

def process_user(user):
    validate_user_email(user)
    # ... processing

def update_user(user):
    validate_user_email(user)
    # ... updating

Extract Module

Creates shared modules for file-level duplicates.

Parameterize

Converts similar code with differences into parameterized functions.

📊 Example Output

╔═══════════════════════════════════════════╗
║        RECATOR - Code Refactoring Bot     ║
║     Eliminate Code Duplicates with Ease   ║
╚═══════════════════════════════════════════╝

🔍 Initializing Recator for: /home/user/project
🔎 Analyzing project for duplicates...

📊 Analysis Results:
  • Total files scanned: 45
  • Files parsed: 42
  • Duplicates found: 8

📋 Duplicate Details:
  [1] Type: exact_block
      Files: utils.py, helpers.py, validation.py
      Confidence: 100%
      Lines: 12

  [2] Type: fuzzy
      Files: api_client.py, http_handler.py
      Confidence: 87%
      Lines: 25

🔧 Refactoring Preview:
  • Total actions: 8
  • Estimated LOC reduction: 147
  • Affected files: 12

✅ Done!

🔧 Configuration File

Create a recator.json configuration file:

{
  "min_lines": 4,
  "min_tokens": 30,
  "similarity_threshold": 0.85,
  "languages": ["python", "javascript", "java"],
  "exclude_patterns": [
    "*.min.js",
    "*.min.css",
    "node_modules/*",
    ".git/*",
    "build/*",
    "dist/*"
  ],
  "safe_mode": true
}

Use with: recator /path/to/project --config recator.json

🧪 Examples

See examples/1/ for a minimal TypeScript example with intentionally duplicated blocks:

# From repository root
recator examples/1 --analyze --languages javascript \
  --min-lines 7 --min-tokens 15 \
  --show-snippets --max-show 0 --max-blocks 0 -v

# Interactive refactor preview for selected duplicates
recator examples/1 --refactor --interactive --dry-run --show-snippets

# Compare suppression behavior
recator examples/1 --analyze --languages javascript --min-lines 7 -v
recator examples/1 --analyze --languages javascript --min-lines 7 -v --no-suppress-duplicates

🧩 Duplicate Snippet Display & On-demand Refactor

• Show snippets: Use --show-snippets with --analyze to print representative code blocks for duplicates (e.g., exact blocks or token previews). Control output size with --max-show.
• On-demand refactor: Use --interactive to choose duplicates interactively, or --select 1,3-5 to pass IDs directly. Combine with --refactor and --dry-run for a safe preview. Use --apply --no-dry-run to apply changes where supported.

Tip: Start with stricter thresholds and increase gradually to avoid excessive output on large codebases.

🧱 Portability Notes

Recator uses a pure-Python, stable 64-bit hashing (FNV-1a) to identify identical fragments. This avoids reliance on OpenSSL-backed hashlib algorithms, so it works even in environments where md5/sha* are unavailable.

🏗️ Architecture

recator/
├── __init__.py       # Main Recator class
├── scanner.py        # File scanning and reading
├── analyzer.py       # Code parsing and tokenization
├── detector.py       # Duplicate detection algorithms
├── refactor.py       # Refactoring strategies
└── cli.py           # Command-line interface

📝 Supported Languages

• Python (.py)
• JavaScript/TypeScript (.js, .jsx, .ts, .tsx)
• Java (.java)
• C/C++ (.c, .cpp, .cc, .cxx, .h, .hpp)
• C# (.cs)
• PHP (.php)
• Ruby (.rb)
• Go (.go)
• Rust (.rs)
• Kotlin (.kt)
• Swift (.swift)

⚙️ How It Works

1. Scanning: Traverses project directory to find source files
2. Parsing: Tokenizes and parses code into analyzable structures
3. Detection: Applies multiple algorithms to find duplicates
4. Analysis: Groups and ranks duplicates by confidence
5. Refactoring: Suggests or applies appropriate refactoring strategies
6. Output: Generates modified files or preview reports

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

📄 License

This project is licensed under the Apache License 2.0.

🙏 Acknowledgments

Built using only Python standard library for maximum compatibility and efficiency.

📮 Support

For issues and questions, please open an issue on GitHub.

---

Made with ❤️ for cleaner, more maintainable code