NeuroSurgeon 0.1.0

A toolkit for subnetwork analysis

NeuroSurgeon

NeuroSurgeon is a python toolkit built to enable deep learning researchers to easily uncover and manipulate subnetworks within trained models. NeuroSurgeon provides a simple API to inject differentiable binary masks techniques into linear, attention, and convolution layers in BERT, GPT, ResNet, and ViT-style models within Huggingface Transformers. Differentiable masking has a variety of use cases for deep learning research, such as:

• Pruning to uncover functional subnetworks
  • Relevant Papers:
    • Csordás et al. 2021
    • Lepori et al. 2023
    • Panigrahi et al. 2023
• Subnetwork Probing
  • Relevant Papers:
    • Cao et al. 2021
    • Conmy et al. 2023
• Training with L0 Regularization
  • Relevant Papers:
    • Louizos et al. 2018
    • Savarese et al. 2020

Documentation

Read the NeuroSurgeon Documentation

Tutorial

To get started with NeuroSurgeon, check out the tutorial here. This covers the basic workflow for using NeuroSurgeon to uncover functional subnetworks within a trained model.

Install

NeuroSurgeon requires python 3.9 or higher and several libraries, including Transformers and PyTorch. Installation can be done using PyPi:

pip install NeuroSurgeon

Logo Prompt

NeuroSurgeon's logo was created with the help of DALL-E 2, using the prompt "A cute cartoon robot doctor smiling with a stethoscope".