weaviate-demo-datasets 0.3.0

Unofficial demo datasets for Weaviate

UNOFFICIAL Weaviate demo data uploader

This is an educational project that aims to make it easy to upload demo data to your instance of Weaviate. The target audience is developers learning how to use Weaviate.

Usage

pip install weaviate-demo-datasets

Each dataset includes a default vectorizer configuration for convenience. The target Weaviate instance must include the specified vectorizer module.

Once you instantiate a dataset, you can upload it to Weaviate with the following:

import weaviate_datasets as wd
dataset = wd.JeopardyQuestions1k()  # Instantiate dataset
dataset.upload_dataset(client)  # Pass the Weaviate client instance

Where client is the instantiated weaviate.WeaviateClient object, such as:

import weaviate
import os

client = weaviate.connect_to_local(
    headers={"X-OpenAI-Api-Key": os.getenv("OPENAI_APIKEY")}
)

To use a weaviate.Client object, as used in the Weaviate Python client v3.x, import the dataset class from weaviate_datasets.v3.

import weaviate_datasets.v3_datasets as wd_v3
dataset = wd_v3.JeopardyQuestions1k()  # Instantiate dataset
dataset.upload_dataset(client)  # Pass the Weaviate client instance

Built-in methods

• .upload_dataset(client) - add defined classes to schema, adds objects
• .get_sample() - yields sample data object(s)

Available classes

• Wiki100 (Top 100 Wikipedia articles) (WikiChunk collection)
• WineReviews (50 wine reviews) (WineReview collection)
• JeopardyQuestions1k (1,000 Jeopardy questions & answers, vectorized with OpenAI text-embedding-ada-002) (JeopardyQuestion and JeopardyCategory collections)
• JeopardyQuestions10k (10,000 Jeopardy questions & answers, vectorized with OpenAI text-embedding-ada-002) (JeopardyQuestion and JeopardyCategory collections)

Available classes - V3 collection

Not including vectors

• WikiArticles (A handful of Wikipedia summaries)
• WineReviews (50 wine reviews)
• WineReviewsMT (50 wine reviews, multi-tenancy enabled)

Including vectors

• JeopardyQuestions1k (1,000 Jeopardy questions & answers, vectorized with OpenAI text-embedding-ada-002)
• JeopardyQuestions1kMT (1,000 Jeopardy questions & answers, multi-tenancy enabled, vectorized with OpenAI text-embedding-ada-002)
• JeopardyQuestions10k (10,000 Jeopardy questions & answers, vectorized with OpenAI text-embedding-ada-002)
• NewsArticles (News articles, including their corresponding publications, authors & categories, vectorized with OpenAI text-embedding-ada-002)

Data sources

https://www.kaggle.com/datasets/zynicide/wine-reviews https://www.kaggle.com/datasets/tunguz/200000-jeopardy-questions https://github.com/weaviate/DEMO-NewsPublications

Source code

https://github.com/databyjp/wv_demo_uploader