histotuner

Supported token-extraction backends

histotuner can append multiple model-specific token tables into the same SpatialData Zarr while keeping shared geometry layers model-agnostic.

Currently supported token extractors:

  • hf-hub:bioptimus/H-optimus-1
  • hf-hub:MahmoodLab/UNI2-h
  • hf-hub:paige-ai/Virchow2
  • hf-hub:Wangyh/mSTAR
  • hf-hub:prov-gigapath/prov-gigapath
  • owkin/phikon-v2
  • MahmoodLab/conchv1_5
  • WenchuanZhang/Patho-CLIP-L
  • majiabo/GPFM

Token-grid semantics

All currently supported models export a unified 14x14 token grid so token tables can be compared directly across models.

  • phikon-v2 exports a native 14x14 patch-token grid.
  • hf-hub:bioptimus/H-optimus-1, hf-hub:Wangyh/mSTAR, and hf-hub:prov-gigapath/prov-gigapath export native 14x14 grids.
  • hf-hub:MahmoodLab/UNI2-h and hf-hub:paige-ai/Virchow2 have native 16x16 patch-token grids after special tokens are stripped, and histotuner adaptively average-pools them to 14x14.
  • conchv1_5 is special:
    • the native vision encoder runs at 448x448 with patch16
    • that produces a native 28x28 patch-token grid
    • histotuner average-pools each non-overlapping 2x2 token neighborhood to export a compatibility 14x14 token grid
  • Patho-CLIP-L is also special:
    • the native CLIP-L/14 vision encoder produces a 24x24 patch-token grid at 336x336 input resolution
    • histotuner adaptively average-pools that native 24x24 grid to export a compatibility 14x14 token grid
  • GPFM is also special:
    • the native DINOv2 ViT-L/14 encoder produces a 16x16 patch-token grid at 224x224 input resolution
    • histotuner adaptively average-pools that native 16x16 grid to export a compatibility 14x14 token grid

That pooling choice is deliberate so downstream single-cell workflows can consume every supported model through the same 14x14 token layout. For the pooled models, this is a compatibility semantic rather than the model's native tokenization:

  • UNI2-h and Virchow2: pooled from native 16x16
  • conchv1_5: pooled from native 28x28
  • Patho-CLIP-L: pooled from native 24x24
  • GPFM: pooled from native 16x16

Not yet supported for token extraction

  • none from the current requested set

O2 batch job generation

To generate one embedder.yaml and one embed_cluster.sh per sample folder on O2:

python generate_o2_jobs.py \
  --root-dir /n/scratch/users/a/ajn16/histotuner/full \
  --template-yaml embedder.yaml \
  --template-shell embed_cluster.sh \
  --output-dir /n/scratch/users/a/ajn16/histotuner/generated_jobs

  python generate_o2_jobs.py \
  --root-dir /n/scratch/users/a/ajn16/histotuner/heonly \
  --template-yaml embedder_HEonly.yaml \
  --template-shell embed_cluster_HEonly.sh \
  --output-dir /n/scratch/users/a/ajn16/histotuner/generated_jobs

To preview the sbatch submissions for the generated job scripts:

python submit_generated_jobs.py \
  --generated-dir /n/scratch/users/a/ajn16/histotuner/generated_jobs \
  --dry-run

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.4.9)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ajitjohnson from gravatar.com ajitjohnson

Credits

Author: Ajit Johnson Nirmal

Requires

Python >=3.12

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

histotuner-0.3.2.tar.gz (162.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

histotuner-0.3.2-py3-none-any.whl (175.3 kB view details)

Uploaded Python 3

File details

Details for the file histotuner-0.3.2.tar.gz.

File metadata

  • Download URL: histotuner-0.3.2.tar.gz
  • Upload date:
  • Size: 162.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.8.13

File hashes

Hashes for histotuner-0.3.2.tar.gz
Algorithm Hash digest
SHA256 18317b3e5ede199283c450aaa041d404c440e43c8bf8393bf9dbef61ad416646
MD5 8889f5bf84ab342268b03d71d00be516
BLAKE2b-256 1d467ba6c9a480f30aa65a6baea2f44003bfd98647a82f995b4e89911803fe65

See more details on using hashes here.

File details

Details for the file histotuner-0.3.2-py3-none-any.whl.

File metadata

  • Download URL: histotuner-0.3.2-py3-none-any.whl
  • Upload date:
  • Size: 175.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.8.13

File hashes

Hashes for histotuner-0.3.2-py3-none-any.whl
Algorithm Hash digest
SHA256 20cffddcfefe3d222f4e16551b7f1794174963a499dc134289cd7d96fadeae89
MD5 022d6f0e2c4fab0b8348f589195e4844
BLAKE2b-256 80d3c90ad890de59a6cd8c94bf59895b1739b07e535995b216867087e0fbec3f

See more details on using hashes here.

Release history Release notifications | RSS feed

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.23

2 files

0.3.22

2 files

0.3.21

2 files

0.3.20

2 files

0.3.19

2 files

0.3.18

2 files

0.3.17

2 files

0.3.16

2 files

0.3.15

2 files

0.3.14

2 files

0.3.13

2 files

0.3.12

2 files

0.3.11

2 files

0.3.10

2 files

0.3.9

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

This release

0.3.2 This release

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.1.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page