teradatagenai 20.0.0.2

Teradata package for Generative-AI powered text analytics on Teradata Vantage

Teradata Package for Generative-AI

teradatagenai is a Generative AI package developed by Teradata. It offers a comprehensive suite of APIs designed for a wide range of text analytics applications and seamless access to the Enterprise Vector Store. With teradatagenai, users can seamlessly process and analyze text data from various sources, including emails, academic papers, social media posts, and product reviews. This enables users to gain insights with precision and depth that rival or surpass human analysis.

For community support, please visit the Teradata Community.

For Teradata customer support, please visit Teradata Support.

Copyright 2025, Teradata. All Rights Reserved.

Table of Contents

• Documentation
• Release Notes
• Installation and Requirements
• Using the Teradata Package for Generative AI
• License

Documentation

General product information, including installation instructions, is available in the Teradata Documentation website.

Release Notes

Version 20.00.00.02

• New Features/Functionality

  • Vector Store

    • Exposes a new attribute store_type in VectorStore class allowing user to check the type of Vector Store. Supported vector store types are metadata-based, content-based, file-based and embedding-based.
    • Exposes following new functions in VectorStore class.
      • get_indexes_embeddings - Returns DataFrame containing embedding and indexing information of the Vector Store.
      • get_model_info - Returns model specific DataFrame or dict containing DataFrames depending on the search_algorithm.
        • If search_algorithm is kmeans, dict is returned containing the two tables mentioned below:
          • kmeans_model - Contains the kmeans_model information.
          • centroids_model - Contains the centroids information.
        • If search_algorithm is hnsw, DataFrame is returned containing:
          • hnsw_model - Contains the hnsw_model information.
        • similarity_search_by_vector - Performs similarity_search for 'embeddings-based' Vector Store when question is embedded and passed in question argument. or embedded question is present in a table and that is passed in data and column arguments.
    • Exposes the following new parameters for create and update:
      • extract_infographics
      • extract_method
      • hf_access_token

  • TextAnalyticsAI Functions

    • Support added in TeradataAI for a new API type 'onnx' to handle external ONNX models within Teradata Vantage.
    • Support added in TextAnalyticsAI to generate Text Embeddings with ONNX models.
    • Support added in TeradataAI and TextAnalyticsAI to work with NVidia NIM, to perform a wide array of text analytic tasks including:
      • KeyPhrase Extraction
      • PII (Personally Identifiable Information) Entity Recognition
      • Masking PII Information
      • Language Detection
      • Language Translation
      • Text Summarization
      • Entity Recognition
      • Sentiment Analysis
      • Text Classification
      • Text Embeddings
      • Asking LLM

• Bug Fixes

  • Response code is not shown for errors raised for async operations like create, update, destroy from status.

Version 20.00.00.01

• New Features/Functionality

  • Vector Store

    • Teradata Enterprise Vector Store is designed to store, index, and search high-dimensional vector embeddings efficiently.
    • teradatagenai provides the below python APIs to easily access and manage vector store and build their own NL applications using Vantage as the foundational compute/storage engine.
      • The following operations can be done:
        • VSManager: Contains methods to manage vector Store.
          • health: Perform health check for the vector store service.
          • list: List all the vector stores.
          • list_sessions: List all the active sessions of the vector store service.
          • disconnect: Disconnect from the database session.
          • list_patterns: List all available patterns for creating metadata-based vector store.
        • VectorStore: Contains methods to do operations on Vector Store.
          • create: Creates a Vector Store.
          • update: Updates a Vector Store.
          • destroy: Destorys a Vector Store.
          • similarity_search: Performs similarity search in interactive/batch mode in the Vector Store for the input question
          • prepare_response: Prepare a natural language response to the user using the input question and similarity_results provided by VectorStore.similarity_search() method using interactive/batch mode.
          • ask: Performs similarity search in the vector store for the input question followed by preparing a natural language response to the user using interactive/batch mode.
          • get_details: Get details of the vector store.
          • get_objects: Get the list of objects in the metadata-based vector store.
          • get_batch_results: Retrieves the results when similarity_search, prepare_response and ask is triggered in batch mode.
          • status: Checks the status of the below operations: create, destroy and update.
        • VSPattern: Create/Manage patterns which provides a way to select tables/views and columns using simple regular expressions which can be used while creating metadata-based Vector Store.
          • Following operations are supported:
            • create: Creates a pattern by specifying the pattern_string.
            • get: Gets the list of objects that matches the pattern_string.
            • delete: Deletes the pattern.

  • InDb TextAnalytics Functions

    • This version supports the integration of TextAnalyticsAI InDB functions, enabling seamless access to LLM services like AWS Bedrock, Azure OpenAI, Google Gemini for a wide array of text analytics tasks, including:
      • KeyPhrase Extraction
      • PII (Personally Identifiable Information) Entity Recognition
      • Masking PII Information
      • Language Detection
      • Language Translation
      • Text Summarization
      • Entity Recognition
      • Sentiment Analysis
      • Text Classification
      • Text Embeddings
      • Asking LLM

Version 20.00.00.00

• teradatagenai 20.00.00.00 marks the first release of the package.
• This version supports the integration of Hugging Face models into Teradata Vantage through the BYO LLM offering, enabling seamless utilization of these models for a wide array of text analytics tasks.
  • KeyPhrase Extraction
  • PII (Personally Identifiable Information) Entity Recognition
  • Masking PII Information
  • Language Detection
  • Language Translation
  • Text Summarization
  • Entity Recognition
  • Sentiment Analysis
  • Text Classification
  • Text Embeddings
  • Sentence Similarity
• The package also features a versatile task function capable of performing any task supported by the underlying language model (LLM). This function is highly adaptable and can be customized to meet specific requirements. Refer to the example for more details on its usage.

Installation and Requirements

Package Requirements:

• Python 3.9 or later

Note: 32-bit Python is not supported.

Minimum System Requirements:

• Windows 7 (64Bit) or later
• macOS 10.9 (64Bit) or later
• Red Hat 7 or later versions
• Ubuntu 16.04 or later versions
• CentOS 7 or later versions
• SLES 12 or later versions
• VantageCloud Lake on AWS with Open Analytics Framework in order to use Teradata’s BYO LLM offering.

Minimum Database Requirements

• Teradata Vantage with database release 20.00 or later

Installation

Use pip to install the Teradata Package for Generative AI

Platform	Command
macOS/Linux	pip install teradatagenai
Windows	python -m pip install teradatagenai

When upgrading to a new version of the teradatagenai, you may need to use pip install's --no-cache-dir option to force the download of the new version.

Platform	Command
macOS/Linux	pip install --no-cache-dir -U teradatagenai
Windows	python -m pip install --no-cache-dir -U teradatagenai

Using the Teradata Package for Generative AI:

Your Python script must import the teradatagenai package in order to use the Teradata Package for Generative AI. Let us walkthrough some examples to gain a better understanding. We need a common setup to load the data and import the required packages.

Common Setup

# Import the modules and create a teradataml DataFrame.
import os
import teradatagenai
from teradatagenai import TeradataAI, TextAnalyticsAI, load_data
from teradataml import DataFrame

load_data('employee', 'employee_data')
data = DataFrame('employee_data')
df_reviews = data.select(["employee_id", "employee_name", "reviews"])
df_articles = data.select(["employee_id", "employee_name", "articles"])

# Define the base directory and script path.
base_dir = os.path.dirname(teradatagenai.__file__)
sentence_similarity_script = os.path.join(base_dir, 'example-data', 'sentence_similarity.py')

Analyze Sentiment of Food Reviews

In this example, we will be using the analyze_sentiment API to analyze the sentiment of food reviews in the reviews column of a teradataml DataFrame.

Using the Hugging Face model distilbert-base-uncased-emotion.

# Define the model name and arguments for the Hugging Face model.
model_name = 'bhadresh-savani/distilbert-base-uncased-emotion'
model_args = {
    'transformer_class': 'AutoModelForSequenceClassification',
    'task': 'text-classification'
}

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="hugging_face", model_name=model_name, model_args=model_args)

# Create a TextAnalyticsAI object.
obj = TextAnalyticsAI(llm=llm)
obj.analyze_sentiment(column='reviews', data=df_reviews, delimiter="#")

Using AWS Bedrock model anthropic.claude-v2.

# Define AWS Bedrock environment variables.
os.environ["AWS_DEFAULT_REGION"] = "<Enter AWS Region>"
os.environ["AWS_ACCESS_KEY_ID"] = "<Enter AWS Access Key ID>"
os.environ["AWS_SECRET_ACCESS_KEY"] = "<Enter AWS Secret Key>"
os.environ["AWS_SESSION_TOKEN"] = "<Enter AWS Session key>"

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="aws", model_name="anthropic.claude-v2")

# Create a TextAnalyticsAI object.
obj = TextAnalyticsAI(llm=llm)
obj.analyze_sentiment(column='reviews', data=df_reviews, accumulate="reviews")

Using Azure OpenAI model gpt-3.5-turbo.

# Define Azure OpenAI environment variables.
os.environ["AZURE_OPENAI_API_KEY"] = "<azure OpenAI API key>"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://****.openai.azure.com/"
os.environ["AZURE_OPENAI_API_VERSION"] = "2000-11-35"
os.environ["AZURE_OPENAI_DEPLOYMENT_ID"] = "<azure OpenAI engine name>"

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="azure", model_name="gpt-3.5-turbo")

# Create a TextAnalyticsAI object.
obj = TextAnalyticsAI(llm=llm)
obj.analyze_sentiment(column='reviews', data=df_reviews, accumulate="reviews")

Using Google model gemini-1.5-pro-001.

# Define Google Cloud environment variables
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "<gcp access token>"
os.environ["GOOGLE_CLOUD_PROJECT"] = "<gcp project name>"
os.environ["GOOGLE_CLOUD_REGION"] = "us-central1"

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="gcp", model_name="gemini-1.5-pro-001")

# Create a TextAnalyticsAI object.
obj = TextAnalyticsAI(llm=llm)
obj.analyze_sentiment(column='reviews', data=df_reviews, accumulate="reviews")

Using NVIDIA NIM model meta/llama-3.1-8b-instruct.

# Define Azure OpenAI environment variables.
os.environ["NIM_API_KEY"] = "<NVIDIA NIM API key>"

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="nim", api_base = "<nim base url>",model_name="meta/llama-3.1-8b-instruct")

# Create a TextAnalyticsAI object.
obj = TextAnalyticsAI(llm=llm)
obj.analyze_sentiment(column='reviews', data=df_reviews, accumulate="reviews")

Get Embeddings and Similarity Score for Employee Data and Articles

In this example, we will use the task API to perform two tasks: generating embeddings and calculating similarity scores using the Hugging Face model all-MiniLM-L6-v2.

Generate Embeddings for Employee Reviews

We will generate embeddings for employee reviews from the articles column of a teradataml DataFrame using the Hugging Face model all-MiniLM-L6-v2.

# Define the script path for embeddings.
embeddings_script = os.path.join(base_dir, 'example-data', 'embeddings.py')

# Construct the returns argument based on the user script.
returns = OrderedDict([('text', VARCHAR(512))])
_ = [returns.update({"v{}".format(i+1): VARCHAR(1000)}) for i in range(384)]

# Use the task API to generate embeddings.
llm.task(
    column="articles",
    data=df_articles,
    script=embeddings_script,
    returns=returns,
    libs='sentence_transformers',
    delimiter='#'
)

Calculate Similarity Score

We will calculate the similarity score between employee data and articles using the Hugging Face model all-MiniLM-L6-v2.

# Define the model name and arguments for the Hugging Face model.
model_name = 'sentence-transformers/all-MiniLM-L6-v2'
model_args = {
    'transformer_class': 'AutoModelForSequenceClassification',
    'task': 'text-similarity'
}

# Create a TeradataAI object with the specified model.
llm = TeradataAI(api_type="hugging_face", model_name=model_name, model_args=model_args)

# Use the task API to get the similarity score.
llm.task(
    column=["employee_data", "articles"],
    data=data,
    script=sentence_similarity_script,
    libs='sentence_transformers',
    returns={
        "column1": "VARCHAR(10000)",
        "column2": "VARCHAR(10000)",
        "similarity_score": "VARCHAR(10000)"
    },
    delimiter="#"
)

License

Use of the Teradata package for Generative-AI is governed by the License Agreement for the Teradata package for Generative-AI. After installation, the LICENSE and LICENSE-3RD-PARTY.pdf files are located in the teradatagenai directory of the Python installation directory.