No project description provided
Project description
Unstructured Haystack
Unstructured Connectors for Haystack
This is an example Haystack 2.0 integration. It's an integration for Unstructured.io connectors. Please contribute 🚀
The current version has 2 available Unstructured connectors:
- Discord:
UnstructuredDiscordConnector
- GitHub:
UnstructuredGitHubConnector
- Google Drive:
UnstructuredGoogleDriveConnector
How to use in a Haystack 2.0 Pipeline
For example, you can write documents fetched from Discord using the UnstructuredDiscordConnector
:
from haystack.preview import Pipeline
from haystack.preview.components.writers import DocumentWriter
from unstructured_haystack import UnstructuredDiscordConnector
from chroma_haystack import ChromaDocumentStore
# Chroma is used in-memory so we use the same instances in the two pipelines below
document_store = ChromaDocumentStore()
connector = UnstructuredDiscordConnector(api_key="UNSTRUCTURED_API_KEY", discord_token="DISCORD_TOKEN")
indexing = Pipeline()
indexing.add_component("connector", connector)
indexing.add_component("writer", DocumentWriter(document_store))
indexing.connect("connector.documents", "writer.documents")
indexing.run({"connector": {"channels" : "993539071815200889", "period": 3, "output_dir" : "discord-example"}})
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for unstructured_haystack-0.0.7.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e7598fede7948cae34cae53cfbdedc375fe2b10c27a69594c4286e402d0a44f9 |
|
MD5 | fbe18d3e58a0af25e15a8d8324ac0306 |
|
BLAKE2b-256 | d91a225f9c5d1dc145faa3978620bc1a495d312d416e01efa7c9f59a91ac4bed |
Close
Hashes for unstructured_haystack-0.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2f3cfbfd7e2b0024eed88fe1d3312e4a648b204d46596f9ac87204c0e6c8ee36 |
|
MD5 | 7be8e39f35d49b1e6e8713588813c45d |
|
BLAKE2b-256 | ce9d3fb475519d26856e965dc3ce91c3c1b979c8bd378c96ca6872a5591b8d46 |