Skip to main content

Multi-Modality

Med-PaLM

A responsible path to generative AI in healthcare: Unleash the power of Med-PaLM 2 to revolutionize medical knowledge, answer complex questions, and enhance healthcare experiences with accuracy, safety, and equitable practices.

GitHub issues GitHub forks GitHub stars GitHub license Share on Twitter Share on Facebook Share on LinkedIn Discord Share on Reddit Share on Hacker News Share on Pinterest Share on WhatsApp

Med palm

Getting Started

pip install MedPalm

Usage

import torch
from medpalm.model import MedPalm

#usage
img = torch.randn(1, 3, 256, 256)
text = torch.randint(0, 20000, (1, 4096))

model = MedPalm()
output = model(img, text)
print(output.shape)

📝 Note: Modify the examples to suit your data and project needs.

📚 Datasets

Here is a comprehensive markdown table containing metadata and details for all the datasets mentioned in the MED PALM paper

Dataset Modality Description # Training examples # Test examples Tasks
MultiMedBench Multimodal Benchmark for biomedical AI 14 biomedical tasks
MedQA Text US licensing exam questions 10,178 1,273 Question answering
MedMCQA Text Indian medical exam questions 182,822 4,183 Question answering
PubMedQA Text Biomedical literature questions 0 500 Question answering
MIMIC-III Radiology reports Radiology reports for ICU patients 58,405 reports 13,057 reports Report summarization
VQA-RAD Radiology images QA pairs on radiology images 1,797 QA pairs 451 QA pairs Visual question answering
Slake-VQA Radiology images English-Chinese QA pairs 9,849 samples 2,070 samples Visual question answering
Path-VQA Pathology images QA pairs on pathology images 19,755 QA pairs 6,761 QA pairs Visual question answering
MIMIC-CXR Chest X-ray Images and reports 353,542 4,834 Report generation, classification
PAD-UFES-20 Dermatology images Skin lesion smartphone images 1,838 images 460 images Image classification
CBIS-DDSM (mass) Mammography Mammogram mass patches 1,318 images 378 images Image classification
CBIS-DDSM (calcification) Mammography Mammogram calcification patches 1,544 images 326 images Image classification
VinDr-Mammo Mammography Mammogram studies 16,000 images 4,000 images Image classification
PrecisionFDA (training) Genomics Genomic variant images 197,038 images Image classification
PrecisionFDA (evaluation) Genomics Genomic variant images 13,030 images Image classification
Montgomery County Chest X-ray Chest X-rays 0 138 images TB detection evaluation
MIMIC-CXR (human evaluation) Chest X-ray Chest X-ray images and reports 246 cases Human evaluation

💼 Commercial Use-Cases

Med Palm has thousands of potential use cases the 3 below are simple, for more detailed applications check out my new blog article on MedPalm's use in the real world. Click here to learn more

  • Clinical Diagnostics: Combining medical imaging, patient tales 📖, and genes, we're aiming for top-notch diagnostic solutions.

  • Healthcare Research: Dive deep into diverse datasets and discover something new with Med-PaLM by your side! 🤿

  • Telemedicine: Quick, reliable, and remote! 🌍 Med-PaLM's here to revolutionize telehealth.

Contributing to Med Palm 🤖🌟

Help with the todo list!


License

Med-PaLM's is under the MIT license. Check out the details here.

Citation

@misc{2307.14334,
Author = {Tao Tu and Shekoofeh Azizi and Danny Driess and Mike Schaekermann and Mohamed Amin and Pi-Chuan Chang and Andrew Carroll and Chuck Lau and Ryutaro Tanno and Ira Ktena and Basil Mustafa and Aakanksha Chowdhery and Yun Liu and Simon Kornblith and David Fleet and Philip Mansfield and Sushant Prakash and Renee Wong and Sunny Virmani and Christopher Semturs and S Sara Mahdavi and Bradley Green and Ewa Dominowska and Blaise Aguera y Arcas and Joelle Barral and Dale Webster and Greg S. Corrado and Yossi Matias and Karan Singhal and Pete Florence and Alan Karthikesalingam and Vivek Natarajan},
Title = {Towards Generalist Biomedical AI},
Year = {2023},
Eprint = {arXiv:2307.14334},
}

Todo

  • Implement a training script using Huggingface's accelerate and deepspeed

Metadata

Release files for medpalm 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for medpalm 0.2.0
File Size Uploaded
medpalm-0.2.0.tar.gz 27.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for medpalm 0.2.0
File Interpreter ABI Platform
medpalm-0.2.0-py3-none-any.whl Python 3 none any Details

Total release size: 52.3 kB

Release files / medpalm-0.2.0.tar.gz

Download URL medpalm-0.2.0.tar.gz
Size 27.4 kB
Tags Source
SHA-256 checksum
How to use checksums
4153c761b2bb99afe0e8ec8ccd98ad98365f532b4db632e15b816d0b85646238
BLAKE2b-256 checksum
How to use checksums
f0f9f0453ebfe71c436d67affd39d1f3238d747b757a8edce3442089639be444
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0

Release files / medpalm-0.2.0-py3-none-any.whl

Download URL medpalm-0.2.0-py3-none-any.whl
Size 24.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
32a64c5c334b50162e4ae506bfc270587e94d5f61f6cace6ccaf8a038074e4b1
BLAKE2b-256 checksum
How to use checksums
e343932345de07b7173b270a55d7945e27721421b94c3fc82e8c32ae69bca9cd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.3.2 CPython/3.11.0 Darwin/23.3.0

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files

0.1.9

2 release files

0.1.7

2 release files

0.1.5

2 release files

0.1.4

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.0

2 release files

0.0.9

2 release files

0.0.8

2 release files

0.0.7

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page