Skip to main content

A (python) indexer for large collections of json files

Project description

A (python) indexer for large collections of json files. In development and probably not good for general use yet.

Installation

pip install JsonPydexer

Usage

Starting from zero

>>>from JsonPydexer import JsonPydexer
>>># initialize with the root directory containing your json files
>>>jp = JsonPydexer("test_data/3")
>>># index on your keys. provide a list of keynamess. keyname elements can be a string for a
>>># field at root level, or a list of strings for nested fields
>>>jp.index(["id", "name", ["field_in_root", "field_below"]])

Opening an existing index

>>># initialize with the root directory containing your json files and .jp.pkl
>>>jp = JsonPydexer("test_data/3")
>>># get all the files that match a search string for non-unique index
>>>search_string = "alice"
>>>for filename in jp.get_files(["name"], search_string):
>>>    print(filename)
1.json
3.json
>>># get the file matching a search string for a unique index
>>>filename = jp.get_file(["id"], "id2")
>>>print(filename)
2.json

Added, ~~removed, or modified~~ JSON files? Update the index

>>>jp = JsonPydexer("test_data/3")
>>>jp.update()

New files in the directory will be added to the index. Coming soon are: removing files not present in the directory from the index, and checking for modified files in the directory.

Current Limitations

As of 0.3.2, the library is not capable of reliably adding new keys to an existing index. Rather, the existing index file (.jp.pkl in the directory you’ve initialized JP in) should be deleted, and JP should be initialized again with the full list of desired keys.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

JsonPydexer-0.3.3.tar.gz (4.5 kB view details)

Uploaded Source

File details

Details for the file JsonPydexer-0.3.3.tar.gz.

File metadata

  • Download URL: JsonPydexer-0.3.3.tar.gz
  • Upload date:
  • Size: 4.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for JsonPydexer-0.3.3.tar.gz
Algorithm Hash digest
SHA256 117c076b521edf4d9a568b2d72010bf4311801679e243640e87ffd28721fc94b
MD5 38d3a69c621a6d15b8ef496448fb9701
BLAKE2b-256 89b1b665a79e62740f3769c973777c13f4ad9838ce3f02b95578357062c97af2

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page