finest 0.1.4

Overview

📊 Core applications

• 📈 Imputation — recover weak or missing gene signals using HE image context

• 🔬 Resolution — refine Visium spots to sub-spot / single-cell, Visium HD 16-µm bin to 8-µm bin

• 🔗 Discovery — identify fine-grained LR pairs and CCC patterns at super resolution (7 or 8-µm)

🎯 Key capabilities

• 💰 Cost-efficient — leverage existing HE images; no extra sequencing required for imputation

• 🖼️ Morphology-aware — contrastive learning links HE cell morphology to gene expression

• ⚡ Multi-resolution — enhance spot/bin resolution to sub-spot, single-cell, or 8-µm bins

• 🌍 Broad applicability — supports Visium, Visium HD datasets for WSI- or ROI-based analysis

🧠 How it works

FineST follows a four-step pipeline:

1. 🖼️ Step 0 — HE image feature extraction (HIPT / Virchow2)

2. 🔄 Step 1 — Training FineST model on within-spot or 16-µm bin expression

3. 📐 Step 2 — Super-resolution Imputation at sub-spot or single-cell level

4. 💬 Step 3 — Fast Discovery of LR pairs and CCC patterns (SpatialDM + SparseAEH)

Supported ST platforms and image encoders

Conda (recommended)

git clone https://github.com/StatBiomed/FineST.git
conda create --name FineST python=3.8
conda activate FineST
cd FineST
pip install -r requirements.txt

Verify:

python -c "import torch; print(torch.__version__); print(torch.cuda.is_available())"

PyPI

pip install -U FineST

## Alternatively, install from GitHub for latest version:
pip install -U git+https://github.com/StatBiomed/FineST

Note: To run the Jupyter notebook tutorials, register this environment as a kernel:

python -m pip install ipykernel
python -m ipykernel install --user --name=FineST

Download demo datasets

📥 Visium tutorial data (FineST_tutorial_data) is available on Google Drive.

pip install gdown
gdown --folder https://drive.google.com/drive/folders/1rZ235pexAMVvRzbVZt1ONOu7Dcuqz5BD?usp=drive_link

📥 Visium HD demo data (FineST_tutorial_data_VisiumHD) from 10x Genomics - Sample P2 CRC.

From the package root (FineST/), create the data folder, download, extract, and arrange files to match run_CRC_VisiumHD_HIPT.sh:

## 1) Create data root next to the package scripts
mkdir -p FineST_tutorial_data_VisiumHD
cd FineST_tutorial_data_VisiumHD

## 2) Download 10x Visium HD (P2 CRC) files
BASE=https://cf.10xgenomics.com/samples/spatial-exp/3.0.0/Visium_HD_Human_Colon_Cancer_P2
wget ${BASE}/Visium_HD_Human_Colon_Cancer_P2_tissue_image.btf
wget ${BASE}/Visium_HD_Human_Colon_Cancer_P2_spatial.tar.gz
wget ${BASE}/Visium_HD_Human_Colon_Cancer_P2_binned_outputs.tar.gz

## 3) Extract archives
tar -xzf Visium_HD_Human_Colon_Cancer_P2_binned_outputs.tar.gz
tar -xzf Visium_HD_Human_Colon_Cancer_P2_spatial.tar.gz

## 4) Arrange layout expected by FineST CLI / run_CRC_VisiumHD_HIPT.sh
mkdir -p square_016um
cp binned_outputs/square_016um/spatial/tissue_positions.parquet square_016um/
cp binned_outputs/square_016um/spatial/scalefactors_json.json square_016um/
mv Visium_HD_Human_Colon_Cancer_P2_tissue_image.btf \
   Visium_HD_Human_Colon_Cancer_tissue_image.btf

cd ..

Expected layout after step 4:

FineST_tutorial_data_VisiumHD/
├── Visium_HD_Human_Colon_Cancer_tissue_image.btf   # HE image (data root)
├── square_016um/
│   ├── tissue_positions.parquet
│   └── scalefactors_json.json
├── binned_outputs/          # from 10x extract (kept; optional after copy)
└── ...                      # optional: spatial/ from spatial.tar.gz

Experienced bioinformatics users

🚀 Command-line demos (from the package root)

For Visium (NPC, HIPT) — ~10 min

bash run_NPC_tutorial_HIPT.sh

• Reproduces NPC_Train_Impute_count_HIPT.ipynb (Sections 0–5)

• DATA_ROOT default: FineST_tutorial_data (download above)

• Outputs under {DATA_ROOT}/{Figures,OrderData,SaveData}/

• Evaluation (infer/impute vs measured spots) on by default; RUN_EVAL=0 to skip

For Visium HD (CRC16, HIPT) — longer; needs data layout above

## First run: extract HE embeddings (~hours), then train/infer/eval
RUN_STEP0=1 bash run_CRC_VisiumHD_HIPT.sh

## Later runs: skip Step 0 if FineST_tutorial_data_VisiumHD/HIPT/ already exists
bash run_CRC_VisiumHD_HIPT.sh

• DATA_ROOT default: FineST_tutorial_data_VisiumHD (replaces notebook FineST_local/Dataset/CRC16um/)

• Expression: FineST.datasets.CRC16um() / CRC08um() (Figshare; auto on first run)

• Weights: finetune/20260801162414255436/

• Embeddings: RUN_STEP0=1 writes HIPT/HD_CRC_16um_pth_32_16/; or place precomputed and keep RUN_STEP0=0

• Evaluation (vs 16 µm input + native 8 µm) on by default; RUN_EVAL=0 to skip

Bioinformatics beginners

⚡ Jupyter Notebook tutorials (recommended first run)

🧬 Visium end-to-end (~10 min)

• HIPT: NPC_Train_Impute_count_HIPT.ipynb

• Virchow2: NPC_Train_Impute_count_Virchow2.ipynb

🗺️ Visium HD end-to-end (~1–3 hours, large data)

• HIPT: CRC16_Train_Impute_count_HIPT.ipynb

• Virchow2: CRC16_Train_Impute_count_virchow2.ipynb

💬 LR / CCC discovery (after imputation)

• Visium: NPC_LRI_CCC_count.ipynb

• Visium HD: CRC_LRI_CCC_count.ipynb

✂️ ROI-based analysis (~1 min)

• ROI selection and cropping: Crop_ROI_Boundary_image.ipynb

Visium (NPC demo)

• Imputation + 8µm enhancement (HIPT): NPC_Train_Impute_count_HIPT.ipynb

• Imputation + 7µm enhancement (Virchow2): NPC_Train_Impute_count_Virchow2.ipynb

• Between-spot interpolation: Between_spot_demo.ipynb

• LR pair & CCC discovery: NPC_LRI_CCC_count.ipynb

• Cell-type deconvolution: transDeconv_NPC_count.ipynb

• Performance evaluation: NPC_Evaluate.ipynb

Visium (HCC P1T demo)

• Imputation + 7µm enhancement (Virchow2): HCC_P1T_Train_Impute.ipynb

Visium HD (CRC 16µm demo)

• Imputation + 8µm enhancement (HIPT): CRC16_Train_Impute_count_HIPT.ipynb

• Imputation + 7µm enhancement (Virchow2): CRC16_Train_Impute_count_virchow2.ipynb

• LR pair & CCC discovery: CRC_LRI_CCC_count.ipynb

• Cell-type deconvolution: transDeconv_CRC_count.ipynb

⚙️ Step 0: HE image feature extraction

🖼️ Visium — within-spots

## HIPT (recommended)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data/spatial/tissue_positions_list.csv \
   --rawimage_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --dataset_class Visium \
   --STfactor_path FineST_tutorial_data/spatial/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model HIPT \
   --patch_size 64 \
   --data_save_dir FineST_tutorial_data

## Virchow2 (requires Hugging Face token)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data/spatial/tissue_positions_list.csv \
   --rawimage_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --dataset_class Visium \
   --STfactor_path FineST_tutorial_data/spatial/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model Virchow2 \
   --patch_size 112 \
   --data_save_dir FineST_tutorial_data

🗺️ Visium HD — 16-µm bins

CLI demo root: FineST_tutorial_data_VisiumHD/ (same as run_CRC_VisiumHD_HIPT.sh). Notebooks may use FineST_local/Dataset/CRC16um/ with the same relative layout under square_016um/.

## HIPT (recommended)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data_VisiumHD/square_016um/tissue_positions.parquet \
   --rawimage_path FineST_tutorial_data_VisiumHD/Visium_HD_Human_Colon_Cancer_tissue_image.btf \
   --dataset_class VisiumHD \
   --STfactor_path FineST_tutorial_data_VisiumHD/square_016um/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model HIPT \
   --patch_size 32 \
   --output_pth FineST_tutorial_data_VisiumHD/HIPT/HD_CRC_16um_pth_32_16 \
   --output_img FineST_tutorial_data_VisiumHD/HIPT/HD_CRC_16um_pth_32_16_image

## Virchow2 (requires Hugging Face token)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data_VisiumHD/square_016um/tissue_positions.parquet \
   --rawimage_path FineST_tutorial_data_VisiumHD/Visium_HD_Human_Colon_Cancer_tissue_image.btf \
   --dataset_class VisiumHD \
   --STfactor_path FineST_tutorial_data_VisiumHD/square_016um/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model Virchow2 \
   --patch_size 28 \
   --output_pth FineST_tutorial_data_VisiumHD/Virchow2/HD_CRC_16um_pth_28_14 \
   --output_img FineST_tutorial_data_VisiumHD/Virchow2/HD_CRC_16um_pth_28_14_image

FineST standardizes image resolution to 0.5 µm/pixel before patch extraction.

Recommended: pass --is_05umperpix True with --STfactor_path (path to scalefactors_json.json) and --dataset_class Visium or VisiumHD. FineST reads Space Ranger scale factors, sets scale_image=True, and computes --scale automatically.

Formula: --scale = microns_per_pixel / 0.5

• Visium (NPC demo)

  • microns_per_pixel = 55 / spot_diameter_fullres from scalefactors_json.json

  • Example (FineST_tutorial_data/spatial/): spot_diameter_fullres = 139.45 → 55 / 139.45 ≈ 0.394 µm/px → --scale ≈ 0.789

• Visium HD (CRC 16 µm)

  • Read microns_per_pixel directly from scalefactors_json.json

  • Example (FineST_tutorial_data_VisiumHD/square_016um/): 0.274 µm/px → --scale = 0.274 / 0.5 ≈ 0.548

Manual alternative: omit --is_05umperpix and set --scale_image True with an explicit --scale value.

🧠 Step 1: Training FineST model

🖼️ Visium — within-spots

Run from the package root (same as run_NPC_tutorial_HIPT.sh).

## HIPT with Visium16 (patch_size=64)
python -m FineST.step1_FineST_train_infer \
   --system_path './' \
   --data_root FineST_tutorial_data \
   --parame_path 'parameter/parameters_NPC_HIPT.json' \
   --dataset_class 'Visium16' \
   --hist_model 'HIPT' \
   --gene_selected 'CD70' \
   --visium_path 'FineST_tutorial_data/spatial/tissue_positions_list.csv' \
   --image_embed_path 'FineST_tutorial_data/ImgEmbeddings/HIPT/pth_64_16' \
   --patch_size 64 \
   --do_scale True \
   --weight_w 0.5

## Virchow2 with Visium64 (patch_size=112)
python -m FineST.step1_FineST_train_infer \
   --system_path './' \
   --data_root FineST_tutorial_data \
   --parame_path 'parameter/parameters_NPC_virchow2.json' \
   --dataset_class 'Visium64' \
   --hist_model 'Virchow2' \
   --gene_selected 'CD70' \
   --visium_path 'FineST_tutorial_data/spatial/tissue_positions_list.csv' \
   --image_embed_path 'FineST_tutorial_data/ImgEmbeddings/Virchow2/pth_112_14' \
   --patch_size 112 \
   --do_scale True \
   --weight_w 0.5

🗺️ Visium HD — 16-µm bins

Same layout as run_CRC_VisiumHD_HIPT.sh. Pretrained HIPT weights: finetune/20260801162414255436/.

## HIPT with VisiumHD (patch_size=32)
python -m FineST.step1_FineST_train_infer \
   --system_path './' \
   --data_root FineST_tutorial_data_VisiumHD \
   --parame_path 'parameter/parameters_CRC16_HIPT.json' \
   --dataset_class 'VisiumHD' \
   --hist_model 'HIPT' \
   --gene_selected 'SPP1' \
   --visium_path 'FineST_tutorial_data_VisiumHD/square_016um/tissue_positions.parquet' \
   --image_embed_path 'FineST_tutorial_data_VisiumHD/HIPT/HD_CRC_16um_pth_32_16' \
   --patch_size 32 \
   --do_scale True \
   --weight_w 0.5 \
   --weight_save_path 'finetune/20260801162414255436'

## Virchow2 with VisiumHD (patch_size=28)
python -m FineST.step1_FineST_train_infer \
   --system_path './' \
   --data_root FineST_tutorial_data_VisiumHD \
   --parame_path 'parameter/parameters_CRC16_virchow2.json' \
   --dataset_class 'VisiumHD' \
   --hist_model 'Virchow2' \
   --gene_selected 'SPP1' \
   --visium_path 'FineST_tutorial_data_VisiumHD/square_016um/tissue_positions.parquet' \
   --image_embed_path 'FineST_tutorial_data_VisiumHD/Virchow2/HD_CRC_16um_pth_28_14' \
   --patch_size 28 \
   --do_scale True \
   --weight_w 0.5

Key parameters

• Must match Step 0

  • --dataset_class — sub-spot tiling: Visium16 (HIPT, 16 tiles), Visium64 (Virchow2, 64 tiles), VisiumHD (Visium HD)

  • --hist_model — image encoder: HIPT (384-dim) or Virchow2 (1280-dim); must match Step 0

• Imputation blending (shown in commands above; adjust as needed)

  • --do_scale (CLI default False; demos / scripts use True) — z-score expression before combining image-inferred (adata_infer) and neighbor-smoothed (adata_smooth) signals

  • --weight_w (default 0.5) — blend weight: adata_imput = weight_w × adata_infer + (1 - weight_w) × adata_smooth

• Auto-inferred (usually omit from command line)

  • With --data_root, fills OrderData/, Figures/, SaveData/ and related paths (same layout as notebook Section 1.2).

  • Without --data_root, output directories are derived from --image_embed_path.

  • LR genes default to the bundled human list: --LRgene_path 'LR_genes'

  • Users can specify the LR gene file explicitly, e.g.: --LRgene_path 'FineST/datasets/LR_gene/LRgene_CellChatDB_baseline_human.csv'

📐 Step 2: Super-resolution imputation

For Visium (~5k spots; 55-µm spot diameter; 100-µm center-to-center distance), enhance spatial resolution at sub-spot (geometric segmentation) or single-cell (nuclei segmentation with StarDist) level.

## Interpolate spots in horizontal and vertical directions
python -m FineST.spot_interpolation \
   --position_path FineST_tutorial_data/spatial/tissue_positions_list.csv

## HIPT (recommended)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data/spatial/tissue_positions_list_add.csv \
   --rawimage_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --dataset_class Visium \
   --STfactor_path FineST_tutorial_data/spatial/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model HIPT \
   --patch_size 64 \
   --data_save_dir FineST_tutorial_data \
   --output_name NEW_pth_64_16

## Virchow2 (requires Hugging Face token)
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data/spatial/tissue_positions_list_add.csv \
   --rawimage_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --dataset_class Visium \
   --STfactor_path FineST_tutorial_data/spatial/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model Virchow2 \
   --patch_size 112 \
   --data_save_dir FineST_tutorial_data \
   --output_name NEW_pth_112_14

## HIPT with Visium16
python -m FineST.step2_High_resolution_impute \
   --system_path './' \
   --data_root FineST_tutorial_data \
   --hist_model HIPT \
   --parame_path 'parameter/parameters_NPC_HIPT.json' \
   --dataset_class 'Visium16' \
   --gene_selected 'CD70' \
   --visium_path 'FineST_tutorial_data/spatial/tissue_positions_list.csv' \
   --imag_within_path 'FineST_tutorial_data/ImgEmbeddings/HIPT/pth_64_16' \
   --imag_betwen_path 'FineST_tutorial_data/ImgEmbeddings/HIPT/NEW_pth_64_16' \
   --weight_save_path 'FineST_tutorial_data/Figures/weights[timestamp]'

## Virchow2 with Visium64
python -m FineST.step2_High_resolution_impute \
   --system_path './' \
   --data_root FineST_tutorial_data \
   --hist_model Virchow2 \
   --parame_path 'parameter/parameters_NPC_virchow2.json' \
   --dataset_class 'Visium64' \
   --gene_selected 'CD70' \
   --visium_path 'FineST_tutorial_data/spatial/tissue_positions_list.csv' \
   --imag_within_path 'FineST_tutorial_data/ImgEmbeddings/Virchow2/pth_112_14' \
   --imag_betwen_path 'FineST_tutorial_data/ImgEmbeddings/Virchow2/NEW_pth_112_14' \
   --weight_save_path 'FineST_tutorial_data/Figures/weights[timestamp]'

## Explicit paths
python -m FineST.nuclei_segmentation \
   --adata_path FineST_tutorial_data/SaveData/adata_imput_all_spot.h5ad \
   --image_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --prob_thresh 0.75 \
   --save_folder NPC_allspot_p075 \
   --out_dir FineST_tutorial_data/NucleiSegments

## Or with path presets (``--save_folder`` still required)
python -m FineST.nuclei_segmentation \
   --data_root FineST_tutorial_data \
   --save_folder NPC_allspot_p075 \
   --prob_thresh 0.75

## HIPT
python -m FineST.image_feature_extraction \
   --position_path FineST_tutorial_data/NucleiSegments/NPC_allspot_p075/position_all_tissue_sc.csv \
   --rawimage_path FineST_tutorial_data/20210809-C-AH4199551.tif \
   --dataset_class Visium \
   --STfactor_path FineST_tutorial_data/spatial/scalefactors_json.json \
   --is_05umperpix True \
   --hist_model HIPT \
   --patch_size 16 \
   --data_save_dir FineST_tutorial_data \
   --output_name sc_pth_16_16

python -m FineST.step2_High_resolution_impute \
   --system_path './' \
   --data_root FineST_tutorial_data \
   --hist_model HIPT \
   --parame_path 'parameter/parameters_NPC_HIPT.json' \
   --dataset_class 'VisiumSC' \
   --gene_selected 'CD70' \
   --image_embed_path_sc 'FineST_tutorial_data/ImgEmbeddings/HIPT/sc_pth_16_16' \
   --weight_save_path 'FineST_tutorial_data/Figures/weights[timestamp]'

💬 Step 3: Fine-grained ligand-receptor interaction

Identify ligand-receptor interactions and communication patterns based on SpatialDM and SparseAEH.

• Visium: NPC_LRI_CCC_count.ipynb

• Visium HD: CRC_LRI_CCC_count.ipynb

Perform cell-type deconvolution on super-resolved gene expression data with expDeconv() from TransImpute.

• Visium: transDeconv_NPC_count.ipynb

• Visium HD: transDeconv_CRC_count.ipynb