kerasformers 1.0.9

KerasFormers: Open-source Keras 3 collection of pretrained models

KerasFormers 🚀

📖 Introduction

KerasFormers is a collection of models with pretrained weights, built entirely with Keras 3. It supports a range of tasks, including classification, object detection (DETR, RT-DETR, RT-DETRv2, RF-DETR, D-FINE, OWL-ViT, OWLv2, Grounding DINO), segmentation (SAM, SAM2, SAM3, SegFormer, DeepLabV3, EoMT, MaskFormer, Mask2Former, OneFormer, MobileViT-DeepLabV3, RF-DETR), monocular depth estimation (Depth Anything V1, Depth Anything V2), feature extraction (DINO, DINOv2, DINOv3), vision-language modeling (CLIP, SigLIP, SigLIP2, MetaCLIP 2), speech recognition (Whisper, Speech2Text, Moonshine), speech-aware language modeling (Granite Speech, Granite Speech Plus), text encoding and masked language modeling (BERT, RoBERTa, XLM-RoBERTa, DeBERTa, DeBERTa-v2, DeBERTa-v3), text generation with large language models (GPT, GPT-2, Qwen2, Qwen2-MoE, Qwen3, Qwen3-MoE, Qwen3.5, Qwen3.5-MoE, GPT-OSS, Llama 2, Llama 3, Llama 4, Mistral, Mixtral, Gemma, Gemma 2, Gemma 4, MiniMax-Text-01, MiniMax-M2, DeepSeek-V2, DeepSeek-V3, DeepSeek-V4, Cohere/Command-R, Cohere2, Cohere2-MoE, GLM-4, GLM-4-0414, GLM-4.5/GLM-4.6, GLM-5/GLM-5.1/GLM-5.2), multimodal vision-language generation (Qwen2-VL, Qwen2.5-VL, Qwen3-VL, InternVL3, Gemma 3, Mistral 3, DeepSeek-VL, Janus-Pro, MiniMax-M3-VL, Cohere2-Vision, GLM-4V, GLM-4.5V), vision-language grounding across object detection, OCR, pointing, and referring (LocateAnything), and more. It includes hybrid architectures like MaxViT alongside traditional CNNs and pure transformers. kerasformers includes custom layers and backbone support, providing flexibility and efficiency across various applications. For backbones, there are various weight variants like in1k, in21k, fb_dist_in1k, ms_in22k, fb_in22k_ft_in1k, ns_jft_in1k, aa_in1k, cvnets_in1k, augreg_in21k_ft_in1k, augreg_in21k, and many more.

⚡ Installation

From PyPI (recommended)

pip install -U kerasformers

From Source

pip install -U git+https://github.com/IMvision12/KerasFormers

📑 Documentation

Per-model guides - with architecture notes, usage examples, and available pretrained weights, live in the docs/ folder, one page per model across every supported task (classification, object detection, segmentation, depth estimation, feature extraction, vision-language, speech recognition, text encoding, and language modeling). Classification backbones share a single page since they all follow the same XModel / XImageClassify two-class structure; each other model has its own. Browse docs/ for the complete, always-up-to-date list.

📑 Models

📝 Text Models

• Text Encoders (text → embeddings, masked LM, classification)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  BERT	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	transformers
  RoBERTa	RoBERTa: A Robustly Optimized BERT Pretraining Approach	transformers
  XLM-RoBERTa	Unsupervised Cross-lingual Representation Learning at Scale	transformers
  DeBERTa	DeBERTa: Decoding-enhanced BERT with Disentangled Attention	transformers
  DeBERTa-v2	DeBERTa: Decoding-enhanced BERT with Disentangled Attention	transformers
  DeBERTa-v3	DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing	transformers

• Text LLMs (text → text)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  Qwen2	Qwen2 Technical Report	transformers
  Qwen2-MoE	Qwen2 Technical Report	transformers
  Qwen3	Qwen3 Technical Report	transformers
  Qwen3-MoE	Qwen3 Technical Report	transformers
  Qwen3.5	Qwen3 Technical Report	transformers
  Qwen3.5-MoE (Qwen3-Next)	Qwen3 Technical Report	transformers
  GPT-OSS	openai/gpt-oss-20b	transformers
  GPT	Improving Language Understanding by Generative Pre-Training	transformers
  GPT-2	Language Models are Unsupervised Multitask Learners	transformers
  Llama 2	Llama 2: Open Foundation and Fine-Tuned Chat Models	transformers (gated)
  Llama 3	The Llama 3 Herd of Models	transformers (gated)
  Llama 4	meta-llama/Llama-4-Scout-17B-16E	transformers (gated)
  Mistral	Mistral 7B	transformers (gated)
  Mixtral	Mixtral of Experts	transformers (gated)
  Cohere (Command-R)	CohereForAI/c4ai-command-r-v01	transformers (gated)
  Cohere2 (Command-R7B / Command-A)	CohereLabs/c4ai-command-r7b-12-2024	transformers (gated)
  Cohere2-MoE	Command Mixture-of-Experts	transformers (gated)
  DeepSeek-V2	DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model	transformers
  DeepSeek-V3	DeepSeek-V3 Technical Report	transformers
  DeepSeek-V4	deepseek-ai/DeepSeek-V4-Flash	transformers
  Gemma	Gemma: Open Models Based on Gemini Research and Technology	transformers (gated)
  Gemma 2	Gemma 2: Improving Open Language Models at a Practical Size	transformers (gated)
  Gemma 4	google/gemma-4-12B	transformers
  MiniMax-Text-01	MiniMax-01: Scaling Foundation Models with Lightning Attention	transformers
  MiniMax-M2	MiniMaxAI/MiniMax-M2	transformers
  GLM-4 (GLM-4-9B)	THUDM/glm-4-9b-chat-hf	transformers
  GLM-4-0414	THUDM/GLM-4-9B-0414	transformers
  GLM-4.5 / GLM-4.6 (MoE)	zai-org/GLM-4.5	transformers
  GLM-5 / GLM-5.1 / GLM-5.2 (MoE)	zai-org/GLM-5.2	transformers

👁️ Vision Models

• Backbones

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  CaiT	Going deeper with Image Transformers	timm
  ConvMixer	Patches Are All You Need?	timm
  ConvNeXt	A ConvNet for the 2020s	timm
  ConvNeXt V2	ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders	timm
  DeiT	Training data-efficient image transformers & distillation through attention	timm
  DenseNet	Densely Connected Convolutional Networks	timm
  EfficientFormer	EfficientFormer: Vision Transformers at MobileNet Speed	timm
  EfficientNet	EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks	timm
  EfficientNet-Lite	EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks	timm
  EfficientNetV2	EfficientNetV2: Smaller Models and Faster Training	timm
  FlexiViT	FlexiViT: One Model for All Patch Sizes	timm
  InceptionNeXt	InceptionNeXt: When Inception Meets ConvNeXt	timm
  Inception-ResNet-v2	Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning	timm
  Inception-v3	Rethinking the Inception Architecture for Computer Vision	timm
  Inception-v4	Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning	timm
  MaxViT	MaxViT: Multi-Axis Vision Transformer	timm
  MiT	SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers	transformers
  MLP-Mixer	MLP-Mixer: An all-MLP Architecture for Vision	timm
  MobileNetV2	MobileNetV2: Inverted Residuals and Linear Bottlenecks	timm
  MobileNetV3	Searching for MobileNetV3	keras
  MobileViT	MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer	transformers
  MobileViTV2	Separable Self-attention for Mobile Vision Transformers	transformers
  NextViT	Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios	timm
  PiT	Rethinking Spatial Dimensions of Vision Transformers	timm
  PoolFormer	MetaFormer is Actually What You Need for Vision	timm
  Res2Net	Res2Net: A New Multi-scale Backbone Architecture	timm
  ResMLP	ResMLP: Feedforward networks for image classification with data-efficient training	timm
  ResNet	Deep Residual Learning for Image Recognition	timm
  ResNetV2	Identity Mappings in Deep Residual Networks	timm
  ResNeXt	Aggregated Residual Transformations for Deep Neural Networks	timm
  SENet	Squeeze-and-Excitation Networks	timm
  Swin Transformer	Swin Transformer: Hierarchical Vision Transformer using Shifted Windows	timm
  Swin Transformer V2	Swin Transformer V2: Scaling Up Capacity and Resolution	timm
  VGG	Very Deep Convolutional Networks for Large-Scale Image Recognition	timm
  ViT	An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale	timm
  Xception	Xception: Deep Learning with Depthwise Separable Convolutions	timm

• Object Detection

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  D-FINE	D-FINE: Redefine Regression Task of DETRs as Fine-grained Distribution Refinement	transformers
  DETR	End-to-End Object Detection with Transformers	transformers
  RT-DETR	DETRs Beat YOLOs on Real-time Object Detection	transformers
  RT-DETRv2	RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformers	transformers
  RF-DETR	RF-DETR: Neural Architecture Search for Real-Time Detection Transformers	transformers
  OWL-ViT	Simple Open-Vocabulary Object Detection with Vision Transformers	transformers
  OWLv2	Scaling Open-Vocabulary Object Detection	transformers
  Grounding DINO	Marrying DINO with Grounded Pre-Training for Open-Set Object Detection	transformers

• Segmentation

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  DeepLabV3	Rethinking Atrous Convolution for Semantic Image Segmentation	torchvision
  EoMT	Your ViT is Secretly an Image Segmentation Model	transformers
  MaskFormer	Per-Pixel Classification is Not All You Need for Semantic Segmentation	transformers
  Mask2Former	Masked-attention Mask Transformer for Universal Image Segmentation	transformers
  MobileViT	MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer	transformers
  MobileViTV2	Separable Self-attention for Mobile Vision Transformers	transformers
  OneFormer	OneFormer: One Transformer to Rule Universal Image Segmentation	transformers
  RF-DETR	RF-DETR: Neural Architecture Search for Real-Time Detection Transformers	transformers
  SAM	Segment Anything	transformers
  SAM2	SAM 2: Segment Anything in Images and Videos	transformers
  SAM3	SAM 3: Segment Anything with Concepts	transformers (gated)
  SegFormer	SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers	transformers

• Feature Extraction

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  DINO	Emerging Properties in Self-Supervised Vision Transformers	torch.hub
  DINOv2	DINOv2: Learning Robust Visual Features without Supervision	transformers
  DINOv3	DINOv3: Self-Supervised Visual Representation Learning at Scale	transformers (gated)

• Depth Estimation

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  Depth Anything V1	Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data	transformers
  Depth Anything V2	Depth Anything V2	transformers

🖼️ Multimodal Models

• Vision-Language Encoders

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  CLIP	Learning Transferable Visual Models From Natural Language Supervision	transformers
  MetaCLIP 2	MetaCLIP 2: A Worldwide Scaling Recipe	transformers
  SigLIP	Sigmoid Loss for Language Image Pre-Training	transformers
  SigLIP2	SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features	transformers

• Multimodal LLMs (image + text → text)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  Qwen2-VL	Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution	transformers
  Qwen2.5-VL	Qwen2.5-VL Technical Report	transformers
  Qwen3-VL	Qwen3 Technical Report	transformers
  InternVL3	InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models	transformers
  Gemma 3	Gemma 3 Technical Report	transformers (gated)
  Mistral 3	mistralai/Mistral-Small-3.1-24B-Instruct-2503	transformers
  Cohere2-Vision (Command-A Vision)	CohereLabs/command-a-vision-07-2025	transformers (gated)
  DeepSeek-VL	DeepSeek-VL: Towards Real-World Vision-Language Understanding	transformers
  Janus-Pro	Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling	transformers
  MiniMax-M3-VL	MiniMaxAI/MiniMax-M3	transformers
  GLM-4V (GLM-4.1V)	zai-org/GLM-4.1V-9B-Thinking	transformers
  GLM-4.5V (MoE)	zai-org/GLM-4.5V	transformers

• Vision-Language Grounding (object detection, OCR, pointing, referring)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  LocateAnything	LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding	transformers

🔊 Audio Models

• Speech (speech → text)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  Whisper	Robust Speech Recognition via Large-Scale Weak Supervision	transformers
  Speech2Text	fairseq S2T: Fast Speech-to-Text Modeling with fairseq	transformers
  Moonshine	Moonshine: Speech Recognition for Live Transcription and Voice Commands	transformers

• Speech LLMs (audio + text → text)

  🏷️ Model Name	📜 Reference Paper	📦 Source of Weights
  Granite Speech	Granite-speech: open-source speech-aware LLMs with strong English ASR capabilities	transformers
  Granite Speech Plus	ibm-granite/granite-speech-4.1-2b-plus	transformers

📜 License

This project leverages timm and transformers for converting pretrained weights from PyTorch to Keras. For licensing details, please refer to the respective repositories.

• 🔖 kerasformers Code: This repository is licensed under the Apache 2.0 License.

🌟 Credits

• The Keras team for their powerful and user-friendly deep learning framework
• The Transformers library for its robust tools for loading and adapting pretrained models
• The pytorch-image-models (timm) project for pioneering many computer vision model implementations
• All contributors to the original papers and architectures implemented in this library

Citing

BibTeX

@misc{gc2025kerasformers,
  author = {Gitesh Chawda},
  title = {KerasFormers},
  year = {2025},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/IMvision12/KerasFormers}}