msgspec-ext 0.5.1

High-performance settings management and validation library powered by msgspec

📖 Read the announcement on Medium

Features

• ⚡ 7x faster than pydantic-settings - Built on msgspec's high-performance validation
• 🎯 26 built-in validators - Email, URLs, IP addresses, MAC addresses, dates, storage sizes, and more
• 🔧 Drop-in API compatibility - Familiar interface, easy migration from pydantic-settings
• 📦 All msgspec types supported - Full compatibility with msgspec's rich type system
• 🔐 Type-safe - Complete type hints and validation
• 📁 .env support - Fast built-in .env parser (169x faster cached loads)
• 🎨 Nested settings - Support for complex configuration structures
• 🪶 Zero dependencies - Only msgspec required

Installation

Using pip:

pip install msgspec-ext

Using uv (recommended):

uv add msgspec-ext

Quick Start

With BaseSettings (Environment Variables)

from msgspec_ext import BaseSettings, EmailStr, HttpUrl, PositiveInt

class AppSettings(BaseSettings):
    # Basic types (msgspec native support)
    name: str
    debug: bool = False

    # Numeric validators
    port: PositiveInt = 8000  # Must be > 0
    workers: PositiveInt = 4

    # String validators
    admin_email: EmailStr  # RFC 5321 validation
    api_url: HttpUrl  # HTTP/HTTPS only

# Load from environment variables and .env file
settings = AppSettings()

print(settings.name)  # my-app
print(settings.port)  # 8000
print(settings.admin_email)  # admin@example.com

# Serialize to dict
print(settings.model_dump())
# Output: {
#   'name': 'my-app',
#   'debug': False,
#   'port': 8000,
#   'workers': 4,
#   'admin_email': 'admin@example.com',
#   'api_url': 'https://api.example.com'
# }

# Serialize to JSON
print(settings.model_dump_json())
# Output: '{"name":"my-app","debug":false,"port":8000,"workers":4,"admin_email":"admin@example.com","api_url":"https://api.example.com"}'

Set environment variables:

export NAME="my-app"
export ADMIN_EMAIL="admin@example.com"
export API_URL="https://api.example.com"

With msgspec Structs (Direct Usage)

All validators work directly with msgspec structs for JSON/MessagePack serialization:

import msgspec
from msgspec_ext import EmailStr, IPv4Address, ByteSize, PositiveInt, dec_hook, enc_hook

class ServerConfig(msgspec.Struct):
    host: IPv4Address
    port: PositiveInt
    admin_email: EmailStr
    max_upload: ByteSize

# From JSON (use dec_hook for custom type conversion)
config = msgspec.json.decode(
    b'{"host":"192.168.1.100","port":8080,"admin_email":"admin@example.com","max_upload":"50MB"}',
    type=ServerConfig,
    dec_hook=dec_hook
)

print(config.host)  # 192.168.1.100
print(int(config.max_upload))  # 50000000 (50MB in bytes)

# To JSON (use enc_hook to serialize custom types)
json_bytes = msgspec.json.encode(config, enc_hook=enc_hook)

Type Support

msgspec-ext supports all msgspec native types plus 26 additional validators for common use cases.

Built-in msgspec Types

msgspec-ext has full compatibility with msgspec's extensive type system:

• Basic: bool, int, float, str, bytes, bytearray
• Collections: list, tuple, set, frozenset, dict
• Typing: Optional, Union, Literal, Final, Annotated
• Advanced: datetime, date, time, timedelta, UUID, Decimal
• msgspec: msgspec.Raw, msgspec.UNSET (re-exported for convenience)

Plus many more - see the full list in msgspec documentation.

Custom Validators (26 types)

msgspec-ext adds 26 specialized validators for common validation scenarios:

🔢 Numeric Constraints (8 types)

from msgspec_ext import (
    PositiveInt, NegativeInt, NonNegativeInt, NonPositiveInt,
    PositiveFloat, NegativeFloat, NonNegativeFloat, NonPositiveFloat
)

class ServerSettings(BaseSettings):
    port: PositiveInt  # Must be > 0
    offset: NegativeInt  # Must be < 0
    retry_count: NonNegativeInt  # Can be 0 or positive
    balance: NonPositiveFloat  # Can be 0 or negative

🌐 Network & Hardware (4 types)

import msgspec
from msgspec_ext import IPv4Address, IPv6Address, IPvAnyAddress, MacAddress

# With BaseSettings
class NetworkSettings(BaseSettings):
    server_ipv4: IPv4Address  # 192.168.1.1
    server_ipv6: IPv6Address  # 2001:db8::1
    proxy_ip: IPvAnyAddress  # Accepts IPv4 or IPv6
    device_mac: MacAddress  # AA:BB:CC:DD:EE:FF

# Or with msgspec.Struct for API responses
class Device(msgspec.Struct):
    name: str
    ip: IPv4Address
    mac: MacAddress

device = msgspec.json.decode(
    b'{"name":"router-01","ip":"192.168.1.1","mac":"AA:BB:CC:DD:EE:FF"}',
    type=Device,
    dec_hook=dec_hook
)

✉️ String Validators (4 types)

from msgspec_ext import EmailStr, HttpUrl, AnyUrl, SecretStr

class AppSettings(BaseSettings):
    admin_email: EmailStr  # RFC 5321 validation
    api_url: HttpUrl  # HTTP/HTTPS only
    webhook_url: AnyUrl  # Any valid URL scheme
    api_key: SecretStr  # Masked in logs: **********

🗄️ Database & Connections (3 types)

from msgspec_ext import PostgresDsn, RedisDsn, PaymentCardNumber

class ConnectionSettings(BaseSettings):
    database_url: PostgresDsn  # postgresql://user:pass@host/db
    cache_url: RedisDsn  # redis://localhost:6379
    card_number: PaymentCardNumber  # Luhn validation + masking

📁 Path Validators (2 types)

from msgspec_ext import FilePath, DirectoryPath

class PathSettings(BaseSettings):
    config_file: FilePath  # Must exist and be a file
    data_dir: DirectoryPath  # Must exist and be a directory

💾 Storage & Dates (3 types)

import msgspec
from msgspec_ext import ByteSize, PastDate, FutureDate
from datetime import date

# With BaseSettings
class AppSettings(BaseSettings):
    max_upload: ByteSize  # Parse "10MB", "1GB", etc.
    cache_size: ByteSize  # Supports KB, MB, GB, KiB, MiB, GiB
    founding_date: PastDate  # Must be before today
    launch_date: FutureDate  # Must be after today

# Or with msgspec.Struct for configuration files
class StorageConfig(msgspec.Struct):
    max_file_size: ByteSize
    cache_limit: ByteSize
    cleanup_after: int  # days

config = msgspec.json.decode(
    b'{"max_file_size":"100MB","cache_limit":"5GB","cleanup_after":30}',
    type=StorageConfig,
    dec_hook=dec_hook
)
print(int(config.max_file_size))  # 100000000

🎯 Constrained Strings (2 types)

from msgspec_ext import ConStr

class UserSettings(BaseSettings):
    # With constraints
    username: ConStr  # Can use min_length, max_length, pattern

# Usage:
username = ConStr("alice", min_length=3, max_length=20, pattern=r"^[a-z0-9]+$")

Complete Validator List

See examples/06_validators.py and examples/07_advanced_validators.py for complete usage examples.

Use Cases

API Request/Response Validation

import msgspec
from msgspec_ext import EmailStr, HttpUrl, PositiveInt, ByteSize, dec_hook, enc_hook

class CreateUserRequest(msgspec.Struct):
    email: EmailStr
    age: PositiveInt
    website: HttpUrl
    max_storage: ByteSize

class UserResponse(msgspec.Struct):
    id: int
    email: EmailStr
    website: HttpUrl

# Validate incoming JSON
request = msgspec.json.decode(
    b'{"email":"user@example.com","age":25,"website":"https://example.com","max_storage":"1GB"}',
    type=CreateUserRequest,
    dec_hook=dec_hook
)

print(request.email)  # user@example.com
print(request.age)  # 25
print(int(request.max_storage))  # 1000000000

# Serialize response
response = UserResponse(id=1, email=request.email, website=request.website)
json_bytes = msgspec.json.encode(response, enc_hook=enc_hook)
print(json_bytes)
# b'{"id":1,"email":"user@example.com","website":"https://example.com"}'

Configuration Files with Validation

import msgspec
from msgspec_ext import IPv4Address, PositiveInt, PostgresDsn, ByteSize, dec_hook

class ServerConfig(msgspec.Struct):
    host: IPv4Address
    port: PositiveInt
    database_url: PostgresDsn
    max_upload: ByteSize
    workers: PositiveInt = 4

# Load from JSON config file
with open("config.json", "rb") as f:
    config = msgspec.json.decode(f.read(), type=ServerConfig, dec_hook=dec_hook)

print(f"Server: {config.host}:{config.port}")
# Server: 192.168.1.50:8080

print(f"Max upload: {int(config.max_upload)} bytes")
# Max upload: 100000000 bytes

print(f"Workers: {config.workers}")
# Workers: 4

Message Queue Data Validation

import msgspec
from msgspec_ext import EmailStr, IPvAnyAddress, FutureDate, dec_hook, enc_hook

class ScheduledTask(msgspec.Struct):
    task_id: str
    notify_email: EmailStr
    target_server: IPvAnyAddress
    execute_at: FutureDate

# Serialize for queue (MessagePack is faster than JSON)
task = ScheduledTask(
    task_id="task-123",
    notify_email=EmailStr("admin@example.com"),
    target_server=IPvAnyAddress("192.168.1.100"),
    execute_at=FutureDate("2025-12-31")
)
msg_bytes = msgspec.msgpack.encode(task, enc_hook=enc_hook)

# Deserialize from queue
received_task = msgspec.msgpack.decode(msg_bytes, type=ScheduledTask, dec_hook=dec_hook)

Advanced Usage

Environment Variables & .env Files

from msgspec_ext import BaseSettings, SettingsConfigDict

class AppSettings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",  # Load from .env file
        env_prefix="APP_",  # Prefix for env vars
        env_nested_delimiter="__"  # Nested config separator
    )

    name: str
    debug: bool = False
    port: int = 8000

# Loads from APP_NAME, APP_DEBUG, APP_PORT
settings = AppSettings()

.env file:

APP_NAME=my-app
APP_DEBUG=true
APP_PORT=3000
APP_DATABASE__HOST=localhost
APP_DATABASE__PORT=5432

Standalone load_dotenv

from msgspec_ext import load_dotenv

# Load .env file into os.environ (does not override existing vars)
load_dotenv()

# Load from a custom path
load_dotenv("config/.env")

# Override existing environment variables
load_dotenv(".env", override=True)

# Custom encoding
load_dotenv(".env", encoding="latin-1")

Nested Configuration

from msgspec_ext import BaseSettings, SettingsConfigDict, PostgresDsn

class DatabaseSettings(BaseSettings):
    host: str = "localhost"
    port: int = 5432
    name: str = "myapp"
    url: PostgresDsn

class AppSettings(BaseSettings):
    model_config = SettingsConfigDict(
        env_file=".env",
        env_nested_delimiter="__"
    )

    name: str = "My App"
    debug: bool = False
    database: DatabaseSettings

# Loads from DATABASE__HOST, DATABASE__PORT, DATABASE__URL, etc.
settings = AppSettings()

print(settings.name)  # My App
print(settings.database.host)  # localhost
print(settings.database.port)  # 5432

# Full nested dump
print(settings.model_dump())
# Output: {
#   'name': 'My App',
#   'debug': False,
#   'database': {
#     'host': 'localhost',
#     'port': 5432,
#     'name': 'myapp',
#     'url': 'postgresql://user:pass@localhost:5432/myapp'
#   }
# }

Secret Masking

from msgspec_ext import BaseSettings, SecretStr

class AppSettings(BaseSettings):
    api_key: SecretStr
    db_password: SecretStr

settings = AppSettings()

print(settings.api_key)  # **********
print(settings.api_key.get_secret_value())  # actual-secret-key

print(settings.model_dump())
# Output: {'api_key': '**********', 'db_password': '**********'}

print(settings.model_dump_json())
# Output: '{"api_key":"**********","db_password":"**********"}'

Storage Size Parsing

from msgspec_ext import BaseSettings, ByteSize

class StorageSettings(BaseSettings):
    max_upload: ByteSize
    cache_limit: ByteSize

# Environment variables:
# MAX_UPLOAD=10MB
# CACHE_LIMIT=1GB

settings = StorageSettings()
print(int(settings.max_upload))  # 10000000 (10 MB in bytes)
print(int(settings.cache_limit))  # 1000000000 (1 GB in bytes)

print(settings.model_dump())
# Output: {'max_upload': 10000000, 'cache_limit': 1000000000}

Supported units: B, KB, MB, GB, TB, KiB, MiB, GiB, TiB

Date Validation

from msgspec_ext import BaseSettings, PastDate, FutureDate
from datetime import date, timedelta

class EventSettings(BaseSettings):
    founding_date: PastDate  # Must be before today
    launch_date: FutureDate  # Must be after today

# Environment variables:
# FOUNDING_DATE=2020-01-01
# LAUNCH_DATE=2025-12-31

settings = EventSettings()

JSON Parsing from Environment

from msgspec_ext import BaseSettings

class AppSettings(BaseSettings):
    # Automatically parse JSON from environment variables
    features: list[str] = ["auth", "api"]
    limits: dict[str, int] = {"requests": 100}
    config: dict[str, any] = {}

# Environment variable:
# FEATURES=["auth","api","payments"]
# LIMITS={"requests":1000,"timeout":30}

settings = AppSettings()
print(settings.features)  # ['auth', 'api', 'payments']

Why Choose msgspec-ext?

msgspec-ext provides a faster, lighter alternative to pydantic-settings while offering more validators and maintaining a familiar API.

Performance Comparison

Cold start (first load, includes .env parsing):

Warm (cached) (repeated loads in long-running applications):

Benchmarks run on Google Colab. Includes .env parsing, environment variable loading, type validation, and nested configuration. Run benchmark/benchmark_cold_warm.py to reproduce.

Key Advantages

How is it so fast?

msgspec-ext achieves exceptional performance through:

1. Bulk validation: Validates all fields at once in C (via msgspec), not one-by-one in Python
2. Custom .env parser: Built-in fast parser with zero external dependencies (117.5x faster than pydantic)
3. Smart caching: Caches .env files, field mappings, and type information - subsequent loads are 169x faster
4. Zero overhead: Fast paths for common types with minimal Python code

This means:

• 🚀 CLI tools - 7.0x faster startup every invocation
• ⚡ Serverless functions - Lower cold start latency
• 🔄 Long-running apps - Reloading settings takes only 11 microseconds after first load!

Examples

Check out the examples/ directory for comprehensive examples:

• 01_basic_usage.py - Getting started with BaseSettings
• 02_env_prefix.py - Using environment variable prefixes
• 03_dotenv_file.py - Loading from .env files
• 04_advanced_types.py - Optional, lists, dicts, JSON parsing
• 05_serialization.py - model_dump(), model_dump_json(), schema()
• 06_validators.py - String, numeric, path, and database validators (17 types)
• 07_advanced_validators.py - Network, storage, and date validators (8 types)

Contributing

We welcome contributions! Please see CONTRIBUTING.md for guidelines.

License

MIT License - see LICENSE file for details.

Acknowledgments

Built on top of the amazing msgspec library by @jcrist.