Skip to main content

Unstract

No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents

Unstract SDK

The unstract-sdk package helps with developing tools that are meant to be run on the Unstract platform. This includes modules to help with tool development and execution, caching, making calls to LLMs / vectorDBs / embeddings .etc. They also contain helper methods/classes to aid with other tasks such as indexing and auditing the LLM calls.

Installation

  • The below libraries need to be installed to run the SDK
    • Linux

      sudo apt install build-essential pkg-config libmagic-dev
      
    • Mac

      brew install pkg-config libmagic pandoc tesseract-ocr
      

Tools

Create a scaffolding for a new tool

Example

unstract-tool-gen --command NEW --tool-name <name of tool> \
 --location ~/path_to_repository/unstract-tools/ --overwrite false

Supported commands:

  • NEW - Create a new tool

Environment variables required for all Tools

Variable Description
PLATFORM_SERVICE_HOST The host in which the platform service is running
PLATFORM_SERVICE_PORT The port in which the service is listening
PLATFORM_SERVICE_API_KEY The API key for the platform
TOOL_DATA_DIR The directory in the filesystem which has contents for tool execution

Llama Index support

Unstract SDK 0.3.2 uses the following version of Llama Index Version 0.9.28 as on January 14th, 2024

Developing with the SDK

Ensure that you have all the required dependencies and pre-commit hooks installed

uv sync
pre-commit install

Once the changes have been made, it can be tested with Unstract through the following means.

With UV

Specify the SDK as a dependency to a project using a tool like uv by adding the following to your pyproject.toml

dependencies = [
    "unstract-sdk"
]

[dependency-groups]
dev = [
"unstract-sdk"
]

[tool.uv.sources]
unstract-sdk = { path = "${UNSTRACT_SDK_PATH", editable = true  }

With pip

  • If the project is using pip it might be possible to add it as a dependency in requirements.txt
-e /path/to/unstract-sdk

NOTE: Building locally might require the below section to be replaced in the unstract-sdk's build system configuration

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
  • Another option is to provide a git URL in requirements.txt, this can come in handy while building tool docker images. Don't forget to run apt install git within the Dockerfile for this
[tool.uv.sources]
unstract-sdk = { git = "git+https://github.com/Zipstack/unstract-sdk@feature-branch" }
  • Or try installing a local PyPI server and upload / download your package from this server

Additonal dependencies for tool

Tools may need to be backed up by a file storage. unstract.sdk.file_storage contains the required interfaces for the same. fsspec is being used underneath to implement these interfaces. Hence, one can choose to use a file system supported by fsspec for this. However, the required dependencies need to be added in the tool dependency manager. Eg. If the tool is using Minio as the underlying file storage, then s3fs can be added to support it. Similarly, for Google Cloud Storage, gcsfs needs to be added. The following versions are tested in the SDK using unit test cases for the above package.

  gcsfs==2024.10.0
  s3fs==2024.10.0

Documentation generation

Follow this README.md for generating documentation.

Metadata

Release files for unstract-sdk 0.79.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for unstract-sdk 0.79.0
File Size Uploaded
unstract_sdk-0.79.0.tar.gz 2.3 MB Details

Built distribution (wheel)

Table of built distributions (wheels) for unstract-sdk 0.79.0
File Interpreter ABI Platform
unstract_sdk-0.79.0-py3-none-any.whl Python 3 none any Details

Total release size: 2.5 MB

Release files / unstract_sdk-0.79.0.tar.gz

Download URL unstract_sdk-0.79.0.tar.gz
Size 2.3 MB
Tags Source
SHA-256 checksum
How to use checksums
fbf093006702d9fbfdbf28fbd89d097faf1cc9c062c7e661626927179dcc5344
BLAKE2b-256 checksum
How to use checksums
c8a6d52b44d566d5584a4471d9ab1f6eccd760865e90e50c84db6c22c72185d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.6.11

Release files / unstract_sdk-0.79.0-py3-none-any.whl

Download URL unstract_sdk-0.79.0-py3-none-any.whl
Size 270.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
d43a9ab000d70ff6b31fab6a148086013cc9f5c327e5d728a1bdae7f19b3bb78
BLAKE2b-256 checksum
How to use checksums
8246f73ed184d2f1f0d1c9fdd76ccde59c2311728307ed1fc172e75354cb234f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.6.11

Release history Release notifications | RSS feed

This release

0.79.0 This release

2 release files

0.77.3

2 release files

0.77.2

2 release files

0.76.1

2 release files

0.76.0

2 release files

0.73.1

2 release files

0.72.0

2 release files

0.71.1

2 release files

0.70.1

2 release files

0.62.1

2 release files

0.62.0

2 release files

0.60.0

2 release files

0.59.0

2 release files

0.58.0

2 release files

0.53.2

2 release files

0.53.1

2 release files

0.53.0

2 release files

0.52.0

2 release files

0.51.0

2 release files

0.49.0

2 release files

0.48.2

2 release files

0.48.0

2 release files

0.47.0

2 release files

0.46.0

2 release files

0.45.2

2 release files

0.45.1

2 release files

0.45.0

2 release files

0.44.0

2 release files

0.39.0

2 release files

0.38.1

2 release files

0.38.0

2 release files

0.35.0

2 release files

0.34.0

2 release files

0.33.1

2 release files

0.33.0

2 release files

0.32.0

2 release files

0.31.1

2 release files

0.28.0

2 release files

0.27.0

2 release files

0.26.1

2 release files

0.26.0

2 release files

0.23.0

2 release files

0.22.1

2 release files

0.22.0

2 release files

0.21.0

2 release files

0.20.1

2 release files

0.20.0

2 release files

0.19.0

2 release files

0.18.2

2 release files

0.17.1

2 release files

0.17.0

2 release files

0.16.0

2 release files

0.15.2

2 release files

0.15.1

2 release files

0.15.0

2 release files

0.12.1

2 release files

0.12.0

2 release files

0.11.2

2 release files

0.11.1

2 release files

0.11.0

2 release files

0.10.1

2 release files

0.10.0

2 release files

0.9.0

2 release files

0.1.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page