sibylline-scurl 0.1.0

A secure curl wrapper with middleware support and HTML-to-markdown extraction

scurl

A secure curl wrapper with middleware support and HTML-to-markdown extraction.

Installation

pip install scurl

Or with pipx (recommended for CLI tools):

pipx install scurl

Usage

# Fetch a URL and extract clean markdown from HTML
scurl https://example.com

# Raw output (disable response middleware)
scurl --raw https://example.com

# All curl flags work
scurl -H "Accept: application/json" https://api.example.com/data

Features

• SecretDefender: Automatically detects and blocks requests containing exposed secrets/tokens
• TrafilaturaExtractor: Extracts clean markdown from HTML responses
• Middleware System: Composable request and response middleware

Flags

Flag	Description
--raw	Disable all response middleware
--disable <slug>	Disable a middleware by slug (can be repeated)
--enable <slug>	Override a middleware's block (can be repeated)
--list-middleware	List available middleware and their slugs

Middleware Slugs

Slug	Type	Description
secret-defender	Request	Detects and blocks requests containing secrets
trafilatura	Response	Extracts clean markdown from HTML

License

MIT