vectara-agentic 0.1.6

A Python package for creating AI Assistants and AI Agents with Vectara

vectara-agentic

The idea of LLM-based agents is to use the LLM for building AI assistants:

• The LLM is used for reasoning and coming up with a game-plan for how to respond to the user query.
• There are 1 or more "tools" provided to the AI assistant. These tools can be used by the LLM to execute its plan.

vectara-agentic is a Python library that let's you develop powerful AI assistants with Vectara, using Agentic-RAG:

• Based on LlamaIndex Agent framework, customized for use with Vectara.
• Supports the ReAct or OpenAIAgent agent types.
• Includes many tools out of the box (e.g. for finance, legal and other verticals).

Important Links

Documentation: https://vectara.github.io/vectara-agentic-docs/

Getting Started

Prerequisites

• A Vectara account
• A Vectara corpus with an API key
• Python 3.10 (or higher)
• An OpenAI API key specified in your environment as OPENAI_API_KEY. Alternatively you can use Anthropic, TOGETHER.AI, Fireworks AI or GROQ to power the assistant In those cases you need to similarly specify your API keys (see below)

Install vectara-agentic

• pip install vectara-agentic

Create your AI assistant

Creating an AI assistant with vectara-agentic involves the following:

Step 1: Create Vectara RAG tool

First, create an instance of the VectaraToolFactory class as follows:

vec_factory = VectaraToolFactory(vectara_api_key=os.environ['VECTARA_API_KEY'],
                                 vectara_customer_id=os.environ['VECTARA_CUSTOMER_ID'], 
                                 vectara_corpus_id=os.environ['VECTARA_CORPUS_ID'])

The Vectara tool factory has a useful helper function called create_rag_tool which automates the creation of a tool to query Vectara RAG.

For example if my Vectara corpus includes financial information from company 10K annual reports for multiple companies and years, I can use the following:

class QueryFinancialReportsArgs(BaseModel):
    query: str = Field(..., description="The user query. Must be a question about the company's financials, and should not include the company name, ticker or year.")
    year: int = Field(..., description=f"The year. an integer.")
    ticker: str = Field(..., description=f"The company ticker. Must be a valid ticket symbol.")

query_financial_reports = vec_factory.create_rag_tool(
    tool_name = "query_financial_reports",
    tool_description = """
    Given a company name and year, 
    returns a response (str) to a user query about the company's financials for that year.
    When using this tool, make sure to provide a valid company ticker and year. 
    Use this tool to get financial information one metric at a time.
    """,
    tool_args_schema = QueryFinancialReportsArgs,
    tool_filter_template = "doc.year = {year} and doc.ticker = '{ticker}'"
)

Note how QueryFinancialReportsArgs defines the arguments for my tool using pydantic's Field class. The tool_description as well as the description of each argument are important as they provide the LLM with the ability to understand how to use this tool in the most effective way. The tool_filter_template provides the template filtering expression the tool should use when calling Vectara.

You can of course create more than one Vectara tool; tools may point at different corpora or may have different parameters for search or generation.

Step 2: Create Other Tools, as needed

In addition to RAG tools, you can generate a lot of other types of tools the agent can use. These could be mathematical tools, tools that call other APIs to get more information, and much more.

vectara-agentic provides a few tools out of the box:

1. Standard tools:

• summarize_text: a tool to summarize a long text into a shorter summary (uses LLM)
• rephrase_text: a tool to rephrase a given text, given a set of rephrase instructions (uses LLM)

2. Legal tools: a set of tools for the legal vertical, such as:

• summarize_legal_text: summarize legal text with a certain point of view
• critique_as_judge: critique a legal text as a judge, providing their perspective

3. Financial tools: based on tools from Yahoo Finance:

• tools to understand the financials of a public company like: balance_sheet, income_statement, cash_flow
• stock_news: provides news about a company
• stock_analyst_recommendations: provides stock analyst recommendations for a company.

4. database_tools: providing a few tools to inspect and query a database

• list_tables: list all tables in the database
• describe_tables: describe the schema of tables in the database
• load_data: returns data based on a SQL query

You can create your own tool directly from a Python function using the create_tool() method:

def mult_func(x, y):
    return x*y

mult_tool = ToolsFactory().create_tool(mult_func)

More tools coming soon!

Step 3: Create your agent

agent = Agent(
    tools = tools,
    topic = topic_of_expertise,
    custom_instructions = financial_bot_instructions,
    update_func = update_func
)

• tools is the list of tools you want to provide to the agent
• topic is a string that defines the expertise you want the agent to specialize in.
• custom_instructions is an optional string that defines special instructions to the agent
• update_func is a callback function that will be called by the agent as it performs its task The inputs to this function you provide are status_type of type AgentStatusType and msg which is a string.

Note that the Agent type (OPENAI or REACT) is defined as an environment variables VECTARA_AGENTIC_AGENT_TYPE.

For example, for a financial agent we can use:

topic = "10-K financial reports",

financial_bot_instructions = """
    - You are a helpful financial assistant in conversation with a user. Use your financial expertise when crafting a query to the tool, to ensure you get the most accurate information.
    - You can answer questions, provide insights, or summarize any information from financial reports.
    - A user may refer to a company's ticker instead of its full name - consider those the same when a user is asking about a company.
    - When calculating a financial metric, make sure you have all the information from tools to complete the calculation.
    - In many cases you may need to query tools on each sub-metric separately before computing the final metric.
    - When using a tool to obtain financial data, consider the fact that information for a certain year may be reported in the the following year's report.
    - Report financial data in a consistent manner. For example if you report revenue in thousands, always report revenue in thousands.
    """

Configuration

vectara-agentic is using environment variables for a few global configuration

• VECTARA_AGENTIC_AGENT_TYPE: type of agent - REACT or OPENAI (default OPENAI)
• VECTARA_AGENTIC_MAIN_LLM_PROVIDER: agent LLM provider OPENAI, ANTHROPIC, TOGETHER, GROQ, or FIREWORKS (default OPENAI)
• VECTARA_AGENTIC_MAIN_MODEL_NAME: agent model name (default depends on provider)
• VECTARA_AGENTIC_TOOL_LLM_PROVIDER: tool LLM provider OPENAI, ANTHROPIC, TOGETHER, GROQ, or FIREWORKS (default OPENAI)
• VECTARA_AGENTIC_TOOL_MODEL_NAME: tool model name (default depends on provider)

Examples

We have created a few example AI assistants that you can look at for inspiration and code examples:

• Financial Assistant.
• Justice Harvard Teaching Assistant.
• Legal Assistant.

Author

👤 Vectara

• Website: vectara.com
• Twitter: @vectara
• GitHub: @vectara
• LinkedIn: @vectara
• Discord: @vectara

🤝 Contributing

Contributions, issues and feature requests are welcome and appreciated!
Feel free to check issues page. You can also take a look at the contributing guide.

Show your support

Give a ⭐️ if this project helped you!

📝 License

Copyright © 2024 Vectara.
This project is Apache 2.0 licensed.