fetchxml 0.1.1

Lightweight session-based XML fetcher with browser-like behavior.

Here is a complete, production-quality README.md for your open-source package fetchxml.

You can paste this directly into README.md.

---

📦 fetchxml

Lightweight, session-based XML fetcher for Python.

fetchxml provides a clean, reusable way to fetch XML from web endpoints that require:

• Browser-like headers
• Session initialization
• Cookie handling
• Referer validation
• Basic anti-bot protection handling

It abstracts session bootstrapping and retry logic into a simple interface.

---

🚀 Why fetchxml?

Some websites block simple HTTP requests and require:

• A session cookie
• Proper User-Agent
• Referer header
• Basic browser simulation

fetchxml handles this automatically.

Instead of writing repetitive session logic every time, you can do:

from fetchxml import FetchXML

client = FetchXML(base_url="https://example.com")
xml = client.fetch("https://example.com/file.xml")

print(xml)

---

📥 Installation

Option 1 – Install from local project

From the project root (where pyproject.toml is located):

pip install .

For development mode:

pip install -e .

---

Option 2 – Install from PyPI (after publishing)

pip install fetchxml

---

🧠 Basic Usage

1️⃣ Simple XML Fetch

from fetchxml import FetchXML

client = FetchXML()

xml = client.fetch("https://example.com/sample.xml")

print(xml[:500])

Use this when the target site does NOT require session bootstrap.

---

2️⃣ Fetch XML With Session Bootstrap

Some sites require hitting their homepage first to establish cookies.

from fetchxml import FetchXML

client = FetchXML(base_url="https://example.com")

xml = client.fetch("https://example.com/sample.xml")

print(xml)

base_url triggers automatic session initialization.

---

3️⃣ Fetch With Custom Referer

If a specific referer header is required:

xml = client.fetch(
    "https://example.com/sample.xml",
    referer="https://example.com/dashboard"
)

---

⚙️ Configuration Options

When initializing:

client = FetchXML(
    base_url="https://example.com",  # optional
    delay=0.5,                       # delay between requests (seconds)
    timeout=15                       # request timeout (seconds)
)

Parameters

Parameter	Description
base_url	URL used to bootstrap session cookies
delay	Sleep time before each request (default 0.5s)
timeout	Request timeout in seconds (default 15s)

---

🔁 Automatic Retry Behavior

If a request returns HTTP 403, fetchxml will:

1. Attempt to re-bootstrap session (if base_url provided)
2. Retry the request once

If it still fails → exception is raised.

---

❗ Exception Handling

All errors raise:

FetchXMLError

Import it like:

from fetchxml import FetchXMLError

Example:

from fetchxml import FetchXML, FetchXMLError

client = FetchXML(base_url="https://example.com")

try:
    xml = client.fetch("https://example.com/sample.xml")
    print(xml)
except FetchXMLError as e:
    print("Failed to fetch XML:", str(e))

---

🔍 What Triggers FetchXMLError?

• Session bootstrap failure
• Non-200 HTTP response
• Timeout
• Connection error
• Persistent 403 after retry

---

🛡️ Rate Limiting

delay ensures a pause before each request:

client = FetchXML(delay=1.5)

Recommended for:

• Bulk XML downloads
• Respecting server load
• Avoiding bot detection

---

📁 Example: Download and Save XML

from fetchxml import FetchXML

client = FetchXML(base_url="https://example.com")

url = "https://example.com/sample.xml"
xml = client.fetch(url)

with open("sample.xml", "w", encoding="utf-8") as f:
    f.write(xml)

print("Saved successfully.")

---

🔧 Advanced: Reusing One Client for Multiple Files

Best practice for bulk downloads:

from fetchxml import FetchXML

client = FetchXML(base_url="https://example.com")

urls = [
    "https://example.com/file1.xml",
    "https://example.com/file2.xml",
    "https://example.com/file3.xml"
]

for url in urls:
    xml = client.fetch(url)
    print(f"Downloaded {url}")

This reuses the same session and cookies.

---

🧪 Testing Connectivity

You can quickly test a URL:

from fetchxml import FetchXML

client = FetchXML()

try:
    xml = client.fetch("https://example.com/sample.xml")
    print("Success")
except Exception as e:
    print("Error:", e)

---

🏗️ Project Structure

fetchxml/
│
├── fetchxml/
│   ├── __init__.py
│   ├── client.py
│   ├── exceptions.py
│
├── pyproject.toml
├── README.md
└── LICENSE

---

📜 License

MIT License

See LICENSE file for full text.

---

⚠️ Disclaimer

fetchxml does not bypass authentication systems or CAPTCHAs.

It simply mimics normal browser session behavior using:

• Session cookies
• Proper headers
• Referer validation

Users are responsible for complying with website terms of service.

---

💡 When To Use fetchxml

Use it when:

• A site blocks naive requests.get()
• Cookies must be initialized first
• Referer headers are required
• You want clean, reusable XML fetching logic

Do NOT use it for:

• Circumventing login walls
• Bypassing paywalls
• Evading legal restrictions

---

🧩 Roadmap (Optional Future Enhancements)

• Async version
• Disk caching layer
• Proxy support
• Built-in XML validation
• Exponential backoff strategy
• Logging integration

---

👤 Author

Saurabh Kumar Agarwal 2026

---