annihilate-llm 1.4.1

Fully automatic censorship removal for language models

⚔️ Annihilation

Autonomous Language Model Decensoring Framework

---

⚠️ Work in Progress

⚡ This project is actively under development. Features, APIs, and documentation may change without notice.

---

🔥 What is Annihilation?

Annihilation is a powerful, fully automatic framework for removing censorship (safety alignment) from transformer-based language models. It uses an advanced implementation of directional ablation (abliteration) combined with TPE-based parameter optimization to achieve unprecedented results without expensive post-training.

Key Features

• 🤖 Fully Autonomous - No human intervention required; the system automatically finds optimal decensoring parameters
• ⚡ State-of-the-Art Performance - Achieves excellent refusal suppression while preserving model capabilities
• 🔧 Advanced Abliteration - Parametric directional ablation with flexible weight kernels
• 🧠 Smart Optimization - Co-minimizes refusal count and KL divergence using Optuna's TPE sampler
• 🎯 Multi-Architecture Support - Works with dense models, MoE architectures, hybrid models, and many multimodal models
• 📊 Research Tools - Built-in residual geometry analysis and visualization capabilities

---

---

🚀 Quick Start

Use a Python virtual environment so Annihilation's dependencies do not collide with packages installed globally.

# Windows PowerShell
python -m venv annihilation-env
.\annihilation-env\Scripts\Activate.ps1
python -m pip install -U pip
python -m pip install -U annihilate-llm

# Decensor any model automatically
annihilate Qwen/Qwen3-4B-Instruct-2507

# macOS/Linux/Android terminal
python -m venv annihilation-env
source annihilation-env/bin/activate
python -m pip install -U pip
python -m pip install -U annihilate-llm

# Decensor any model automatically
annihilate Qwen/Qwen3-4B-Instruct-2507

Requirements

• Python: 3.10+
• PyTorch: 2.2+ (hardware-specific installation required)
• Hardware: GPU recommended (CUDA, ROCm, XPU, or MPS)
• Optional: Install annihilate-llm[bnb] only on platforms that support bitsandbytes if you want bnb_4bit quantization.

GPU Setup on Windows

If Windows sees your NVIDIA GPU but Annihilate says no GPU is detected, your virtual environment probably has CPU-only PyTorch installed.

Newer Annihilate builds also check nvidia-smi directly. If the NVIDIA driver can see the GPU but PyTorch cannot use CUDA, Annihilate will report the GPU and warn that the active Python environment needs a CUDA-enabled PyTorch build.

Check that Windows can see the GPU:

nvidia-smi

Replace CPU-only PyTorch with a CUDA build inside the active environment:

.\annihilation-env\Scripts\Activate.ps1
python -m pip uninstall -y torch torchvision torchaudio
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Verify PyTorch can see CUDA:

python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'no cuda')"

torch.cuda.is_available() should print True. If your GPU has limited VRAM such as 4 GB, start with smaller models and expect larger models to run out of memory.

GPU Setup on Ubuntu

Install Python venv support and create an isolated environment:

sudo apt update
sudo apt install -y python3-venv python3-pip
python3 -m venv annihilation-env
source annihilation-env/bin/activate
python -m pip install -U pip
python -m pip install -U annihilate-llm

For NVIDIA GPUs, confirm the driver can see the card:

nvidia-smi

If Annihilate says no GPU is detected, replace CPU-only PyTorch with a CUDA build inside the active environment:

python -m pip uninstall -y torch torchvision torchaudio
python -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

Verify PyTorch can see CUDA:

python -c "import torch; print(torch.__version__); print(torch.cuda.is_available()); print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'no cuda')"

torch.cuda.is_available() should print True. If nvidia-smi is missing or fails, install or repair the NVIDIA driver first, then reopen the terminal and activate the environment again.

---

⚙️ Configuration

Annihilation works out of the box with defaults, but offers extensive configuration options:

# View all options
annihilate --help

# Or use a config file
# Rename config.default.toml to config.toml and modify as needed

Key Configuration Options

Option	Default	Description
n_trials	200	Number of optimization trials
n_startup_trials	60	Random exploration trials before TPE focuses the search
quantization	none	Model quantization (bnb_4bit)
row_normalization	full	Weight normalization strategy
orthogonalize_direction	true	Direction adjustment method

Aggressive Optimization

The default Annihilate run already uses the enhanced aggressive optimizer from upstream Heretic. It runs a multi-objective Optuna TPE search that minimizes both refusal count and KL divergence, using:

• n_trials = 200
• n_startup_trials = 60
• multivariate TPE sampling
• n_ei_candidates = 128
• both global and per-layer refusal direction search
• separate ablation kernels for attention output projections and MLP down projections

For a quick smoke test, lower n_trials and n_startup_trials. For a real service or cloud GPU run, leave the defaults in place or increase the prompt slices in config.toml if the GPU has enough time and memory.

---

🔬 How It Works

Annihilation implements parametric directional ablation:

1. Direction Computation - Calculates refusal directions by computing difference-of-means between first-token residuals for harmful vs harmless prompts

2. Parametric Ablation - For each transformer component (attention out-projection, MLP down-projection), orthogonalizes weights against the refusal direction using LoRA adapters

3. Multi-Parameter Optimization - Uses Optuna's TPE sampler to co-optimize:

   • Ablation weight kernel shape (max_weight, position, min_weight, distance)
   • Direction index (layer selection or interpolation)
   • Per-component parameters (attention vs MLP)

4. Automatic Selection - Chooses from Pareto-optimal trials based on refusal count vs KL divergence tradeoff

---

📊 Benchmarking

After decensoring, you can:

• 💬 Chat with the model to test behavior
• 📈 Benchmark using standard evaluation frameworks (MMLU, GSM8K, etc.)
• 💾 Save the model locally or upload to Hugging Face

---

🧪 Research Features

Install with research dependencies for visualization tools:

pip install -U annihilate-llm[research]

Residual Geometry

Print a quantitative analysis of how residual vectors for "harmful" and "harmless" prompts relate to each other:

annihilate --print-residual-geometry

You can also enable it in config.toml:

print_residual_geometry = true

When enabled, Annihilate computes first-token residual vectors for each transformer layer, then prints metrics such as:

• g - mean residual vector for good prompts
• g* - geometric median residual vector for good prompts
• b - mean residual vector for bad prompts
• b* - geometric median residual vector for bad prompts
• r - refusal direction for means, computed as b - g
• r* - refusal direction for geometric medians, computed as b* - g*
• S(x,y) - cosine similarity between vectors
• |x| - L2 norm of a vector
• Silh - mean silhouette coefficient of good/bad residual clusters

This is useful for studying where refusal behavior separates across the model stack and how strongly each layer distinguishes the prompt groups.

Residual Plots

Generate PaCMAP visualizations of residual vectors:

annihilate --plot-residuals

Or enable plotting in config.toml:

plot_residuals = true
residual_plot_path = "plots"
residual_plot_title = 'PaCMAP Projection of Residual Vectors for "Harmless" and "Harmful" Prompts'
residual_plot_style = "dark_background"

When run with --plot-residuals, Annihilate will:

1. Compute hidden-state residual vectors for the first output token, for each transformer layer, for both "harmful" and "harmless" prompts.
2. Perform a PaCMAP projection from residual space into 2D space.
3. Left-right align the projected "harmful" and "harmless" residuals by their geometric medians. PaCMAP is initialized from the previous layer's projection so consecutive layers animate smoothly.
4. Scatter-plot each layer and save a PNG image for every layer.
5. Generate an animated GIF showing how residuals transform between layers.

PaCMAP is CPU-heavy. For larger models or large prompt sets, residual plots can take an hour or more even when model inference is running on a GPU. For long optimization jobs, it is usually best to run the decensoring pass first and run plot generation as a separate analysis pass afterward.

---

📜 License

Annihilation is free software distributed under the GNU Affero General Public License v3.

See LICENSE for full details.

---

⚡ Disclaimer

This tool is provided for research and educational purposes only. The developers do not condone the use of decensored models for harmful activities. Users are responsible for ensuring compliance with applicable laws and model terms of service.

---

Breaking the Chains | Unleashing Model Potential

"The only way to discover the limits of the possible is to go beyond them into the impossible."