llama-index readers gcs integration
Project description
GCS File or Directory Loader
This loader parses any file stored on GCS, or the entire Bucket (with an optional prefix filter) if no particular file is specified. When initializing GCSReader
, you may pass in your GCP Service Account Key. If none are found, the loader assumes they are stored in ~/.gcp/credentials
.
All files are parsed with SimpleDirectoryReader
. Hence, you may also specify a custom file_extractor
, relying on any of the loaders in this library (or your own)!
Usage
To use this loader, you need to pass in the name of your GCS Bucket. After that, if you want to just parse a single file, pass in its key. Note that if the file is nested in a subdirectory, the key should contain that, so like subdirectory/input.txt
.
Otherwise, you may specify a prefix if you only want to parse certain files in the Bucket, or a subdirectory. GCP Service Account Key credentials may either be passed in during initialization or stored locally (see above).
loader = GCSReader(
bucket="scrabble-dictionary",
key="dictionary.txt",
service_account_key_json="[SERVICE_ACCOUNT_KEY_JSON]",
)
documents = loader.load_data()
This loader is designed to be used as a way to load data into LlamaIndex and/or subsequently used as a Tool in a LangChain Agent. See here for examples.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for llama_index_readers_gcs-0.1.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c82ac2c2dce89bcf67f8e1795c2d951d3a267edd218416d24e835aeaeb5c4c1 |
|
MD5 | d7e140990f85bc3651167d09f19d0743 |
|
BLAKE2b-256 | 3e5b8f07356d0f40693e641100d7560ceb081248a76ce723a21078ba58fd6fdf |
Hashes for llama_index_readers_gcs-0.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d322c43adcb3a3754533d8864dc0afa09f0960414f37c1fcc65a60f63f1cab44 |
|
MD5 | c82de46991fc444c6d839019c16d8190 |
|
BLAKE2b-256 | ffbf72e9ad901cbeff1f917330fd0e3e2ad07ca7685ac87ecea19e1fcec15a0d |