idc-index-data
About
idc-index-data is a Python package providing index files to query and download
data hosted by the
NCI Imaging Data Commons (IDC).
The PyPI package bundles a core set of index files (Parquet, JSON schemas, SQL queries). Supplementary indices that are too large for PyPI distribution are published as release artifacts on GitHub and uploaded to a public Google Cloud Storage bucket on each release.
Index files
The package provides metadata for the following indices via the INDEX_METADATA
dictionary:
| Index | In PyPI package | Description |
|---|---|---|
idc_index |
yes | Core IDC DICOM study-level index |
prior_versions_index |
yes | Historical version tracking |
collections_index |
yes | Collection-level metadata |
analysis_results_index |
yes | Analysis results metadata |
clinical_index |
- | Clinical data (large) |
sm_index |
- | Slide microscopy index (large) |
sm_instance_index |
- | Slide microscopy instance-level index (large) |
seg_index |
- | Segmentation index |
ann_index |
- | Annotation index |
ann_group_index |
- | Annotation group index |
contrast_index |
- | Contrast agent index |
Additionally, the following supplementary parquet files are generated and published alongside the index files (not included in the PyPI package):
| File | Description |
|---|---|
gdc_idc_mapping.parquet |
Mapping of IDC patients to GDC cases |
tcia_idc_subset.parquet |
Subset of IDC index columns for TCIA workflows |
All index files (including supplementary ones) are available from:
- GitHub Releases: attached as release assets
- Google Cloud Storage: publicly readable via HTTPS
Google Cloud Storage artifacts
Artifacts are uploaded to the idc-index-data-artifacts bucket on each release.
Two paths are maintained:
| Path | Description |
|---|---|
gs://idc-index-data-artifacts/<version>/release_artifacts/ |
Artifacts for a specific release (e.g. 23.5.0) |
gs://idc-index-data-artifacts/current/release_artifacts/ |
Always points to the latest release |
Individual files can be accessed via HTTPS at:
https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts/<filename>
For example:
Replace current with a specific version tag (e.g. 23.5.0) to pin to a
particular release.
Integrity verification
Each parquet file is accompanied by a SHA256 checksum sidecar
(<name>.parquet.sha256) published alongside it. Use it to verify a downloaded
file:
curl -O https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts/idc_index.parquet
curl -O https://storage.googleapis.com/idc-index-data-artifacts/current/release_artifacts/idc_index.parquet.sha256
sha256sum -c idc_index.parquet.sha256
Every parquet file also embeds the idc-index-data package version that
generated it as Apache Parquet schema metadata under the key
idc_index_data_version:
import pyarrow.parquet as pq
meta = pq.read_metadata("idc_index.parquet")
print(meta.metadata[b"idc_index_data_version"])
Usage
This package is intended to be used by the idc-index Python package.
import idc_index_data
# Access core index file paths
idc_index_data.IDC_INDEX_PARQUET_FILEPATH
idc_index_data.PRIOR_VERSIONS_INDEX_PARQUET_FILEPATH
# Access unified metadata for all indices
idc_index_data.INDEX_METADATA["idc_index"]["parquet_filepath"]
idc_index_data.INDEX_METADATA["idc_index"]["schema"] # pre-loaded dict
idc_index_data.INDEX_METADATA["idc_index"]["sql"] # pre-loaded string
Acknowledgment
This software is maintained by the IDC team, which has been funded in whole or in part with Federal funds from the NCI, NIH, under task order no. HHSN26110071 under contract no. HHSN261201500003l.
If this package helped your research, we would appreciate if you could cite IDC paper below.
Fedorov, A., Longabaugh, W. J. R., Pot, D., Clunie, D. A., Pieper, S. D., Gibbs, D. L., Bridge, C., Herrmann, M. D., Homeyer, A., Lewis, R., Aerts, H. J. W., Krishnaswamy, D., Thiriveedhi, V. K., Ciausu, C., Schacherer, D. P., Bontempi, D., Pihl, T., Wagner, U., Farahani, K., Kim, E. & Kikinis, R. National Cancer Institute Imaging Data Commons: Toward Transparency, Reproducibility, and Scalability in Imaging Artificial Intelligence. RadioGraphics (2023). https://doi.org/10.1148/rg.230180
3 maintainers
Metadata
3 maintainers
Release files for idc-index-data 24.2.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| idc_index_data-24.2.2.tar.gz | 77.4 MB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| idc_index_data-24.2.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 154.7 MB
Release files / idc_index_data-24.2.2.tar.gz
| Download URL | idc_index_data-24.2.2.tar.gz |
|---|---|
| Size | 77.4 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ca94c5f524a9bc6832556227c2a3bc955037c9ba9125cfc77d94f7a1d92de528
|
|
BLAKE2b-256 checksum How to use checksums |
29a16e0e4a7efb1567d8ac6d03133d2e35863cc9dc92765794b3d0a948e2e248
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 11, 2026.
Transparency logRelease files / idc_index_data-24.2.2-py3-none-any.whl
| Download URL | idc_index_data-24.2.2-py3-none-any.whl |
|---|---|
| Size | 77.3 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
79ac3a43665077b16cc09e16875c9762ce646257ed0c75900a8c636a54456c62
|
|
BLAKE2b-256 checksum How to use checksums |
70b33adf736fa7db08e64dbbc04084a48054c18cc08822bee75ae5d6f7832f1c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.12
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jul 11, 2026.
Transparency log
