Scraping-friendly HTTP, string parsing, JSON helpers, and AI parsing.

These details have not been verified by PyPI

Project description

Respondo

Scraping-friendly Python library for response parsing, text extraction, JSON detection, and AI-powered parsing. Zero external dependencies—standard library only.

Installation

pip install respondo

Quick Example

from respondo import between, find_first_json, extract_links, Response

# Extract text between delimiters
between("<title>Hello World</title>", "<title>", "</title>")
# => "Hello World"

# Find JSON embedded in text
find_first_json('callback({"user": "alice", "id": 42})')
# => {"user": "alice", "id": 42}

# Parse HTTP response
resp = Response(200, {"content-type": "text/html"}, b"<a href='/page'>Link</a>")
resp.is_success()  # => True
extract_links(resp.text, base="https://example.com")
# => ["https://example.com/page"]

Error Handling

All functions return empty values instead of raising exceptions. This is intentional for scraping where missing data is expected.

between("no match here", "<a>", "</a>")  # => ""
find_first_json("not json")              # => None
regex_first("abc", r"\d+")               # => ""
b64_decode("invalid!!!")                 # => b""
json_get({"a": 1}, "b", "c")             # => None

API Reference

Text Extraction

from respondo import between, betweens, between_last, between_n

# First match between delimiters
between("a]x[b]y[c", "[", "]")           # => "b"

# All matches
betweens("a[b]c[d]e", "[", "]")          # => ["b", "d"]

# Last occurrence of left delimiter
between_last("a[b]c[d]e", "[", "]")      # => "d"

# Nth match (1-indexed)
between_n("a[b]c[d]e", "[", "]", 2)      # => "d"

Regex

from respondo import regex_first, regex_all

# First match (returns capture group if present)
regex_first("price: $42.99", r"\$(\d+)")      # => "42"
regex_first("abc123def", r"\d+")              # => "123"

# All matches
regex_all("<a>1</a><a>2</a>", r"<a>(\d+)</a>")  # => ["1", "2"]

HTML Processing

from respondo import (
    strip_tags,
    get_text,
    extract_links,
    extract_forms,
    extract_tables,
    parse_csrf_token,
)

html = """
<html>
  <script>var x=1;</script>
  <body>
    <h1>Title</h1>
    <p>Hello &amp; welcome</p>
    <a href="/page1">Link 1</a>
    <a href="/page2">Link 2</a>
  </body>
</html>
"""

# Strip HTML tags (keeps script/style content)
strip_tags("<b>bold</b> text")           # => "bold text"

# Extract visible text (removes scripts/styles, decodes entities)
get_text(html)                           # => "Title Hello & welcome Link 1 Link 2"

# Extract all links
extract_links(html, base="https://example.com")
# => ["https://example.com/page1", "https://example.com/page2"]

# Filter to same host only
extract_links(html, base="https://example.com", same_host=True)

# Filter by extension
extract_links(html, extensions=[".pdf", ".doc"])

# Extract forms with fields
extract_forms('<form action="/login" method="post"><input name="user"></form>')
# => [{"action": "/login", "method": "post", "fields": {"user": ""}}]

# Extract tables as dicts
extract_tables("<table><tr><th>Name</th></tr><tr><td>Alice</td></tr></table>")
# => [{"headers": ["Name"], "rows": [{"Name": "Alice"}]}]

# Find CSRF tokens
parse_csrf_token('<input name="csrf_token" value="abc123">')  # => "abc123"
parse_csrf_token('<meta name="csrf-token" content="xyz">')    # => "xyz"

JSON Utilities

from respondo import find_first_json, find_all_json, json_get, json_in_html

# Find first valid JSON in text
find_first_json('prefix {"a": 1} middle [1,2,3] suffix')
# => {"a": 1}

# Find all JSON objects/arrays
find_all_json('{"a":1} text [1,2] more {"b":2}')
# => [{"a": 1}, [1, 2], {"b": 2}]

# Safe nested access (returns None if path missing)
data = {"user": {"profile": {"name": "Alice"}}}
json_get(data, "user", "profile", "name")  # => "Alice"
json_get(data, "user", "missing", "key")   # => None

# Array indexing
json_get([{"id": 1}, {"id": 2}], 0, "id")  # => 1

# Extract JSON from HTML script tags
html = '<script type="application/json">{"config": true}</script>'
json_in_html(html)  # => [{"config": true}]

Encoding Utilities

from respondo import url_encode, url_decode, b64_encode, b64_decode

# URL encoding
url_encode({"q": "hello world", "page": 1})  # => "q=hello+world&page=1"
url_encode({"tags": ["a", "b"]})              # => "tags=a&tags=b"

# URL decoding (returns lists for multi-values)
url_decode("a=1&b=2&b=3")  # => {"a": ["1"], "b": ["2", "3"]}

# Base64
b64_encode("hello")           # => "aGVsbG8="
b64_decode("aGVsbG8=")        # => b"hello"

# URL-safe base64
b64_encode("data", urlsafe=True)
b64_decode("ZGF0YQ", urlsafe=True)

Response Class

from respondo import Response

# Create from status, headers, body
resp = Response(
    status=200,
    headers={"Content-Type": "application/json"},
    body=b'{"ok": true}'
)

# Status checks
resp.is_success()       # 200-299
resp.is_redirect()      # 300-399
resp.is_client_error()  # 400-499
resp.is_server_error()  # 500-599

# Headers (case-insensitive)
resp.header("content-type")      # First value: "application/json"
resp.headers_all("set-cookie")   # All values: ["a=1", "b=2"]

# Content type parsing
resp.content_type()  # => ("application/json", "utf-8")

# Body access
resp.body        # Raw bytes
resp.text        # Decoded string (utf-8)
resp.json()      # Parsed JSON

# Hashing
resp.hash()          # SHA-256 of body
resp.hash("md5")     # MD5 of body

# Charset detection (from headers or <meta>)
text, charset = resp.charset_sniff()

# Cookie parsing
resp.cookies()
# => [{"name": "session", "value": "abc", "attrs": {"path": "/", "httponly": ""}}]

# HTML helpers (delegates to htmlutil)
resp.extract_links(base="https://example.com", same_host=True)
resp.extract_json()           # Find JSON in HTML
resp.strip_scripts_styles()   # Clean HTML text

AI Parsing

Parse text using LLM APIs. Supports 10 providers. No SDK dependencies—uses urllib.

from respondo import parse_ai, parse_ai_json, list_providers

# See all available providers
list_providers()
# => {"openai": "gpt-4o-mini", "anthropic": "claude-3-5-haiku-latest", ...}

# Basic parsing
text = "Contact us at support@example.com or sales@example.com"
parse_ai("Extract all email addresses", text, provider="openai")
# => "support@example.com, sales@example.com"

# Custom model
parse_ai("Summarize", text, provider="openai", model="gpt-4o")
parse_ai("Summarize", text, provider="groq", model="llama-3.1-8b-instant")
parse_ai("Summarize", text, provider="anthropic", model="claude-3-5-sonnet-latest")

# Get JSON response
parse_ai_json("Extract name and age as JSON", "John is 30 years old")
# => {"name": "John", "age": 30}

# Structured output with JSON Schema (enforced by provider)
schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "integer"},
        "email": {"type": "string"}
    },
    "required": ["name", "age", "email"],
    "additionalProperties": False
}
parse_ai_json("Extract person info", text, provider="openai", schema=schema)
# => {"name": "John", "age": 30, "email": "john@example.com"}

# Pass API key directly (instead of env variable)
parse_ai("Extract dates", text, provider="gemini", api_key="...")

Providers:

Provider	Env Variable	Default Model
`openai`	`OPENAI_API_KEY`	`gpt-4o-mini`
`anthropic`	`ANTHROPIC_API_KEY`	`claude-3-5-haiku-latest`
`gemini`	`GEMINI_API_KEY`	`gemini-2.0-flash`
`grok`	`GROK_API_KEY`	`grok-2-latest`
`mistral`	`MISTRAL_API_KEY`	`mistral-small-latest`
`groq`	`GROQ_API_KEY`	`llama-3.3-70b-versatile`
`cohere`	`COHERE_API_KEY`	`command-r`
`together`	`TOGETHER_API_KEY`	`meta-llama/Llama-3.3-70B-Instruct-Turbo`
`deepseek`	`DEEPSEEK_API_KEY`	`deepseek-chat`
`perplexity`	`PERPLEXITY_API_KEY`	`sonar`

Returns empty string ("") or None on errors—no exceptions.

Real-World Examples

Scrape Hacker News titles

import urllib.request
from respondo import Response, regex_all, normalize_space

req = urllib.request.urlopen("https://news.ycombinator.com")
resp = Response(req.status, dict(req.headers), req.read())

titles = regex_all(resp.text, r'class="titleline"[^>]*><a[^>]*>([^<]+)</a>')
for title in titles[:5]:
    print(normalize_space(title))

Extract data from JSON API

from respondo import Response, json_get

resp = Response(200, {}, b'{"users": [{"name": "Alice"}, {"name": "Bob"}]}')
data = resp.json()

# Safe traversal
first_user = json_get(data, "users", 0, "name")  # => "Alice"
missing = json_get(data, "users", 99, "name")    # => None (no error)

Parse form and submit

from respondo import extract_forms, parse_csrf_token, url_encode

html = '''
<form action="/login" method="post">
  <input name="csrf" value="token123">
  <input name="username">
  <input name="password" type="password">
</form>
'''

forms = extract_forms(html, base="https://example.com")
# => [{"action": "https://example.com/login", "method": "post", "fields": {"csrf": "token123", "username": "", "password": ""}}]

# Or just grab the CSRF token
csrf = parse_csrf_token(html)  # => "token123"

Running Tests

cd src
python demo_real_sites.py

License

MIT

Project details

These details have not been verified by PyPI

Release history Release notifications | RSS feed

0.4.0

Jan 14, 2026

0.3.1

Jan 13, 2026

This version

0.3.0

Jan 13, 2026

0.2.0

Jan 13, 2026

0.1.0

Jan 13, 2026

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

respondo-0.3.0.tar.gz (16.7 kB view details)

Uploaded Jan 13, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

respondo-0.3.0-py3-none-any.whl (14.2 kB view details)

Uploaded Jan 13, 2026 Python 3

File details

Details for the file respondo-0.3.0.tar.gz.

File metadata

Download URL: respondo-0.3.0.tar.gz
Upload date: Jan 13, 2026
Size: 16.7 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.13.2

File hashes

Hashes for respondo-0.3.0.tar.gz
Algorithm	Hash digest
SHA256	`9b250327ebc6e48f14fba110dfbf336e4248b20bd7dc9ac4488688e26686d5e6`
MD5	`ae14938ebeeeac178befaa3530906505`
BLAKE2b-256	`e535557fe8e499a6ae568029f1ecb73b9a945bf144b4007f240cfc8bc2675a90`

See more details on using hashes here.

File details

Details for the file respondo-0.3.0-py3-none-any.whl.

File metadata

Download URL: respondo-0.3.0-py3-none-any.whl
Upload date: Jan 13, 2026
Size: 14.2 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: twine/6.2.0 CPython/3.13.2

File hashes

Hashes for respondo-0.3.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`2017bdb4ea6830eceb91ec64831dc75907251a6e32ebc2fad080882f7be1686b`
MD5	`907c086578cc5350a964758064fad5cb`
BLAKE2b-256	`c7d5033f424096ca8116ce22737a6ef60cd8a05c1781f31ed1adbe614efd320e`

See more details on using hashes here.

respondo 0.3.0

Navigation

Verified details

Maintainers

Unverified details

Meta

Classifiers

Project description

Respondo

Installation

Quick Example

Error Handling

API Reference

Text Extraction

Regex

HTML Processing

JSON Utilities

Encoding Utilities

Response Class

AI Parsing

Real-World Examples

Scrape Hacker News titles

Extract data from JSON API

Parse form and submit

Running Tests

License

Project details

Verified details

Maintainers

Unverified details

Meta

Classifiers

Release history Release notifications | RSS feed

Download files

Source Distribution

Built Distribution

File details

File metadata

File hashes

File details

File metadata

File hashes