capturekit-haystack
Haystack integration for CaptureKit, a web page API that loads pages in a real browser and returns them as clean Markdown.
CaptureKitFetcher fetches web pages and returns them as Haystack Documents: the content is the page as Markdown, and the metadata holds url, title and description. No HTML converter is needed.
Installation
pip install capturekit-haystack
Get an API key at app.capturekit.dev and set it in the environment:
export CAPTUREKIT_API_KEY="your-api-key"
Usage
from haystack_integrations.components.fetchers.capturekit import CaptureKitFetcher
fetcher = CaptureKitFetcher()
docs = fetcher.run(urls=["https://example.com"])["documents"]
print(docs[0].meta["title"], docs[0].content[:200])
| Parameter | Default | Description |
|---|---|---|
api_key |
Secret.from_env_var("CAPTUREKIT_API_KEY") |
CaptureKit API key |
main_content_only |
True |
Keep the article or product and drop menus, sidebars and footers |
raise_on_failure |
True |
Raise when a single URL fails; with several URLs, failures are logged and skipped |
timeout |
120 |
Seconds to wait for each page |
A failed request raises CaptureKitError, a Haystack ComponentError with the HTTP status. The API key never appears in the message. run_async is available too.
Index pages for RAG
from haystack import Pipeline
from haystack.components.preprocessors import DocumentSplitter
from haystack.components.writers import DocumentWriter
from haystack.document_stores.in_memory import InMemoryDocumentStore
from haystack_integrations.components.fetchers.capturekit import CaptureKitFetcher
store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("fetcher", CaptureKitFetcher())
pipeline.add_component("splitter", DocumentSplitter(split_by="word", split_length=200))
pipeline.add_component("writer", DocumentWriter(document_store=store))
pipeline.connect("fetcher.documents", "splitter.documents")
pipeline.connect("splitter.documents", "writer.documents")
pipeline.run({"fetcher": {"urls": ["https://docs.haystack.deepset.ai/docs/intro"]}})
print(store.count_documents())
Development
pip install -e ".[test]"
pytest
License
MIT
Metadata
Release files for capturekit-haystack 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| capturekit_haystack-0.1.0.tar.gz | 7.1 kB | Without provenance | Details for capturekit_haystack-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| capturekit_haystack-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for capturekit_haystack-0.1.0-py3-none-any.whl |
Total release size: 13.9 kB
Release files / capturekit_haystack-0.1.0.tar.gz
| Download URL | capturekit_haystack-0.1.0.tar.gz |
|---|---|
| Size | 7.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b476acd55319f88f983fdd2e6e146d07ba048a3afc67563794accfd707bb1312
|
|
BLAKE2b-256 checksum How to use checksums |
f2e39ad2eafa7ce94677e2402488d1e8e2dd349164cec7225853427860398ca7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.4
|
Release files / capturekit_haystack-0.1.0-py3-none-any.whl
| Download URL | capturekit_haystack-0.1.0-py3-none-any.whl |
|---|---|
| Size | 6.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
603c2e677fc342035b6862ba0c2fb801ce42d8bc57b825191f1dd9aac5b2c1d5
|
|
BLAKE2b-256 checksum How to use checksums |
09a4f217618da142a9801a700e689549b5af33529c245e2411f740b2aeaf82a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.4
|
Security for capturekit-haystack 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.