scPEFT 0.1.2

Harnessing the Power of Single Cell Large Language Models with Parameter Efficient Fine-Tuning.

scPEFT

This is the official repository for Harnessing the Power of Single Cell Large Language Models with Parameter Efficient Fine-Tuning

 

Installation

scPEFT works with Python >= 3.7.13. Please make sure you have the correct version of Python installed pre-installation.

scPEFT is available on PyPI. To install scPEFT, run the following command:

pip install scpeft

For developing, run the following command:

git clone https://github.com/SELECT-FROM/scPEFT
cd scPEFT

Note: requirements. ymal are all the dependency versions we recommend using. To ensure smooth code operation, please try to keep them consistent with these package versions as much as possible

Get Started

1. Download the upstream model scGPT model checkpoint and place it at e.g., work_dir/scPEFT/save. We recommend using the whole-human model for most applications by default, which pretrained on 33 million normal human cells..

2. The tutorials of scPEFT for downstream tasks in tutorial_peft. Here are the links to the downstream tasks and tutorials mentioned in our article

   Downstream task	Link
   cell type identification	Tutorial_Identification.ipynb
   batch correction	Tutorial_BatchCorrection.ipynb
   perturbation	Tutorial_Perturbation.ipynb
   cell population discovery	Tutorial_CellPopulationDiscovery.ipynb
   marker gene detection	Tutorial_MarkerGeneDetection.ipynb

To-do-list

• Publish to pypi
• Adapting scPEFT for native-attention
• Adapting scPEFT for flash-attention
• Only retain PEFT-related parameters when saving peft-model weights.

Contributing

We greatly welcome contributions to scPEFT. Please submit a pull request if you have any ideas or bug fixes. We also welcome any issues you encounter while using scPEFT.

Built With

We sincerely thank the authors of following open-source projects:

• scGPT
• scanpy
• scib
• pytorch

Citing scPEFT

@article {He2024.01.27.577455,
	author = {Fei He and Ruixin Fei and Mingyue Gao and Li Su and Xinyu Zhang and Dong Xu},
	title = {Parameter-Efficient Fine-Tuning Enhances Adaptation of Single Cell Large Language Model for Cell Type Identification},
	year = {2024},
	doi = {10.1101/2024.01.27.577455},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2024/01/30/2024.01.27.577455},
	journal = {bioRxiv}
}