Skip to main content

llama-index packs llama_dataset_metadata integration

Project description

LlamaDataset Metadata Pack

As part of the LlamaDataset submission package into llamahub, two metadata files are required, namely: card.json and README.md. This pack creates these two files and saves them to disk to help expedite the submission process.

CLI Usage

You can download llamapacks directly using llamaindex-cli, which comes installed with the llama-index python package:

llamaindex-cli download-llamapack LlamaDatasetMetadataPack --download-dir ./llama_dataset_metadata_pack

You can then inspect the files at ./llama_dataset_metadata_pack and use them as a template for your own project!

Code Usage

You can download the pack to the ./llama_dataset_metadata_pack directory through python code as well. The sample script below demonstrates how to construct LlamaDatasetMetadataPack using a LabelledRagDataset downloaded from llama-hub and a simple RAG pipeline built off of its source documents.

from llama_index.core.llama_pack import download_llama_pack

# Download and install dependencies
LlamaDatasetMetadataPack = download_llama_pack(
    "LlamaDatasetMetadataPack", "./llama_dataset_metadata_pack"
)

# construction requires a query_engine, a rag_dataset, and optionally a judge_llm
llama_dataset_metadata_pack = LlamaDatasetMetadataPack()

# create and save `card.json` and `README.md` to disk
dataset_description = (
    "A labelled RAG dataset based off an essay by Paul Graham, consisting of "
    "queries, reference answers, and reference contexts."
)

llama_dataset_metadata_pack.run(
    name="Paul Graham Essay Dataset",
    description=dataset_description,
    rag_dataset=rag_dataset,  # defined earlier not shown here
    index=index,  # defined earlier not shown here
    benchmark_df=benchmark_df,  # defined earlier not shown here
    baseline_name="llamaindex",
)

NOTE: this pack should be used only after performing a RAG evaluation (i.e., by using RagEvaluatorPack on a LabelledRagDataset). In the code snippet above, index, rag_dataset, and benchmark_df are all objects that you'd expect to have only after perfoming the RAG evaluation as mention in the previous sentence.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

File details

Details for the file llama_index_packs_llama_dataset_metadata-0.1.0.tar.gz.

File metadata

File hashes

Hashes for llama_index_packs_llama_dataset_metadata-0.1.0.tar.gz
Algorithm Hash digest
SHA256 66a1b42e7d596fb3cf8daead81c780e0f6b670d76070ddedc45fe87dc49aa858
MD5 67f9cbc66bf7356b0e7fedce87eb8a30
BLAKE2b-256 97e7e6b1a8039aeeb903fde497c0d1be62e3e5c118fcf81f1393ff1ed6e146ae

See more details on using hashes here.

File details

Details for the file llama_index_packs_llama_dataset_metadata-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for llama_index_packs_llama_dataset_metadata-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 bc8b757071d81e3fbba08b0ea34aca764ebc431d779dad8dbd0e3202bdb34083
MD5 877d37e0a98d1fa1f5d987656cca067d
BLAKE2b-256 175689a232fb376f9980983054c18a563214e3d395f42818977ffabe3ae06e94

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page