Skip to main content

"Efficient, consistent and secure library for querying structured data with natural language"

Project description

db-ally

Efficient, consistent and secure library for querying structured data with natural language



db-ally is an LLM-powered library for creating natural language interfaces to data sources. While it occupies a similar space to the text-to-SQL solutions, its goals and methods are different. db-ally allows developers to outline specific use cases for the LLM to handle, detailing the desired data format and the possible operations to fetch this data.

db-ally effectively shields the complexity of the underlying data source from the model, presenting only the essential information needed for solving the specific use cases. Instead of generating arbitrary SQL, the model is asked to generate responses in a simplified query language.

The benefits of db-ally can be described in terms of its four main characteristics:

  • Consistency: db-ally ensures predictable output formats and confines operations to those predefined by developers, making it particularly well-suited for applications with precise requirements on their behavior or data format
  • Security: db-ally prevents direct database access and arbitrary SQL execution, bolstering system safety
  • Efficiency: db-ally hides most of the underlying database complexity, enabling the LLM to concentrate on essential aspects and improving performance
  • Portability: db-ally introduces an abstraction layer between the model and the data, ensuring easy integration with various database technologies and other data sources.

Quickstart

In db-ally, developers define their use cases by implementing views and filters. A list of possible filters is presented to the LLM in terms of IQL (Intermediate Query Language). Views are grouped and registered within a collection, which then serves as an entry point for asking questions in natural language.

This is a basic implementation of a db-ally view for an example HR application, which retrieves candidates from an SQL database:

from dbally import decorators, SqlAlchemyBaseView, create_collection
from dbally.llm_client.openai_client import OpenAIClient
from sqlalchemy import create_engine

class CandidateView(SqlAlchemyBaseView):
    """
    A view for retrieving candidates from the database.
    """

    def get_select(self):
        """
        Defines which columns to select.
        """
        return sqlalchemy.select(Candidate.id, Candidate.name, Candidate.country)

    @decorators.view_filter()
    def from_country(self, country: str):
        """
        Filter candidates from a specific country.
        """
        return Candidate.country == country

engine = create_engine('sqlite:///candidates.db')
llm = OpenAIClient(model_name="gpt-3.5-turbo")
my_collection = create_collection("collection_name", llm)
my_collection.add(CandidateView, lambda: CandidateView(engine))

my_collection.ask("Find candidates from United States")

For a concrete step-by-step example on how to use db-ally, go to Quickstart guide. For a more learning-oriented experience, check our db-ally Tutorial.

Motivation

db-ally was originally developed at deepsense.ai. In our work on various projects, we frequently encountered the need to retrieve data from data sources, typically databases, in response to natural language queries.

The standard approach to this issue involves using the text-to-SQL technique. While this method is powerful, it is also complex and challenging to control. Often, the results were unsatisfactory because the Language Model lacked the necessary context to understand the specific requirements of the application and the business logic behind the data.

This led us to experiment with a more structured approach. In this method, the developer defines the specific use cases for the Language Model to handle, detailing the desired data format and the possible operations to retrieve this data. This approach proved to be more efficient, predictable, and easier to manage, making it simpler to integrate with the rest of the system.

Eventually, we decided to create a library that would allow us to use this approach in a more systematic way, and we made it open-source for the community.

Installation

To install db-ally, execute the following command:

pip install dbally

Additionally, you can install one of our extensions to use specific features.

pip install dbally[openai,faiss,langsmith]

License

db-ally is released under MIT license.

How db-ally documentation is organized

  • Quickstart - Get started with db-ally in a few minutes
  • Concepts - Understand the main concepts behind db-ally
  • How-to guides - Learn how to use db-ally in your projects
  • Tutorials - Follow step-by-step tutorials to learn db-ally
  • API reference - Explore the underlying API of db-ally

Roadmap

db-ally is actively developed and maintained by a core team at deepsense.ai and a community of contributors.

You can find a list of planned features and integrations in the Roadmap.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dbally-0.1.0.tar.gz (50.3 kB view details)

Uploaded Source

Built Distribution

dbally-0.1.0-py2.py3-none-any.whl (68.9 kB view details)

Uploaded Python 2Python 3

File details

Details for the file dbally-0.1.0.tar.gz.

File metadata

  • Download URL: dbally-0.1.0.tar.gz
  • Upload date:
  • Size: 50.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.19

File hashes

Hashes for dbally-0.1.0.tar.gz
Algorithm Hash digest
SHA256 d526a5b95d02b8158725de6c1ba6957698b829b444b74ac9f42fb95113800bd3
MD5 6a9972467fd00a9978f6f461eb3d5c85
BLAKE2b-256 e3f6886a8ab277d0f86ed8ad40b0497d12ba9295a22a05f2c95933832ec6530f

See more details on using hashes here.

File details

Details for the file dbally-0.1.0-py2.py3-none-any.whl.

File metadata

  • Download URL: dbally-0.1.0-py2.py3-none-any.whl
  • Upload date:
  • Size: 68.9 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.8.19

File hashes

Hashes for dbally-0.1.0-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 e67822462fd64ca8bf1dd3e06dc0a602fb3c1158c5aeb292d504070b44e66bba
MD5 a305910c1909bb5aab0b91fcd59e7866
BLAKE2b-256 eeec5b570f1454450c05da4a8cf5c299ad05f4ed9889d7c4a02e261fe72d83f6

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page