Skip to main content

Keep the bloat out! - A lightweight LLM interaction library

Project description

tests

python pre-commit black license: MIT Contributor Covenant

🤺 Fence

Fence is a simple, lightweight library for LLM communication. A lot of the functionality was inspired by/derived of LangChain (the OG LLM package) basics, since that's how the package was born - as a stripped down version of LangChain functionality, with cooler names.

🤔 Raison d'être

The simple answer: by accident. The slightly longer answer: LangChain used to be (is?) a pretty big package with a ton of dependencies. The upside is that it's powerful for PoC purposes, because it has it all.

The downsides:

  • It's big. It takes up a lot of space (which can be an issue in some environments/runtimes), often for functionality that isn't needed.
  • It's fairly complex. It's a big package with a lot of functionality, which can be overwhelming for new users.
  • It wasn't exactly dependable in an industrial setting before. Version jumps were common, and the package was often broken after a new release.

As a result, many developers (particularly those working in large production environments) have advocated for more lightweight, custom functionality that favors stability and robustness.

Circling back: why Fence?

Since our work was in a production environment, mostly dealing with Bedrock, we just started building some basic components from scratch. We needed a way to communicate with our models, which turned out to as the Link class (wink wink). Then, some other things were added left and right, and this eventually turned into a miniature package. Not in small part because it was fun to go down this road. But mostly because it strikes the right balance between convenience and flexiblity.

Naturally, it's nowhere as powerful as, for instance, LangChain. If you want to build a quick PoC with relatively complex logic, maybe go for the OG instead. If you want to be set on your way with a simple, lightweight package that's easy to understand and extend, Fence might be the way to go.

💪 Features

What can I do with Fence?

  • Uniform interface for LLMs. Since our main use case was Bedrock, we built Fence to work with Bedrock models. However, it also has openAI support, and it's easy to extend to other models (contributors welcome!)
  • Links and Chains help you build complex pipelines with multiple models. This is a feature that's been around since LangChain, and it's still here. You can parametrize templates, and pass the output of one model to another.
  • Template classes that handle the basics, and that work across models (e.g., a MessageTemplate can be sent to a Bedrock Claude3 model, or to an openAI model - system/user/assistant formatting is handled under the hood).
  • Basic utils on board for typical tasks like retries, parallelization, logging, output parsers, etc.

What can't I do with Fence?

It's obviously not as powerful as some of the other packages out there, that hold tons more of features. We're also not trying to fall into the trap of building 'yet another framework' (insert XKCD here), so we're trying to guard our scope. If you need a lot of bells and whistles, you might want to look at any of these:

The OG, no explanation needed.

A more recent package, with a lot of cool features! Great for building PoCs, too. Built by ex-AWS folks, and promises to be a lot more industry-oriented.

🛠️ How do I use it?

Fence just has a few basic components. See the notebooks for examples on how to use them. Documentation is coming soon, but for now, you can check out the source code for more details.

📦 Installation

You can install Fence from PyPI:

pip install fence-llm

🗺️ Roadmap

  • Add more models
  • Add more tests
  • Add some basic design patterns (e.g., CoT, MapReduce)
  • Add more tutorials
  • Add more user-friendly auth methods (e.g., SigV4, OpenAI keys as params/config, etc.)

🤝 Contributing

We welcome contributions! Check out the CONTRIBUTING.md for more details.

🙋🏻‍♂️ Open issues or obstacles

  • Bedrock access depends on user logging in themselves ➡️ Can we make this more user-friendly?

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fence_llm-0.0.10.tar.gz (38.6 kB view details)

Uploaded Source

Built Distribution

fence_llm-0.0.10-py3-none-any.whl (55.0 kB view details)

Uploaded Python 3

File details

Details for the file fence_llm-0.0.10.tar.gz.

File metadata

  • Download URL: fence_llm-0.0.10.tar.gz
  • Upload date:
  • Size: 38.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for fence_llm-0.0.10.tar.gz
Algorithm Hash digest
SHA256 ab95514409e63b72cfa60b8eb1628a6fee0cda3c28e8782987792f25e6d46bfd
MD5 2452ac5fbfe4c99a2323bb0bde219bad
BLAKE2b-256 4f9e511df428eaf422fbb05716ebb326ce1b13e1c5c6816ef52b0bc650b9b024

See more details on using hashes here.

File details

Details for the file fence_llm-0.0.10-py3-none-any.whl.

File metadata

  • Download URL: fence_llm-0.0.10-py3-none-any.whl
  • Upload date:
  • Size: 55.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for fence_llm-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 463ece3a052765604857c1f115a736bcab19e82a60ffab52b2a1a575fbab6550
MD5 c3380dcafd3718f4b3706a09470ef2e0
BLAKE2b-256 d6ef775da40f2b733057dc14e862339cc25c190f4a5fef645763ed5f61b8fbe5

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page