Truncated JSON completer
Project description
truncjson
A library to complete truncated JSON strings.
Use Cases
- When you possess a JSON file that is truncated, and you wish to parse its current content.
- When you are receiving JSON content as a stream from services like ChatGPT. This allows you to parse the JSON content each time you receive a new chunk.
Simple Usage
from truncjson import complete
result = complete('{"k1":"v1","k2":20.5,"k3":[12')
print(result)
# Output: {"k1":"v1","k2":20.5,"k3":[12]}
Stream Usage
Here we created a dummy stream with a list of chunks.
from truncjson import Stream
chunks = [
'Here yo',
'u go {"k1":"v1","k2":2',
'0.5,"k3":[12,34,"56",tr',
'ue,{"k31":"v31","k32":false,"k33":n',
'ull}]}\nAnd ano',
'ther one {"k',
'ey":1.',
'12} {"a',
'b":"cd"}'
]
stream = Stream()
for chunk in chunks:
print(stream.extract(chunk))
# Output:
# []
# ['{"k1":"v1","k2":2}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"k":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.0}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"a":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"ab":"cd"}']
ChatGPT Usage
import openai
import json
from truncjson import Stream
openai.api_key = '###'
chat = openai.ChatCompletion.create(
model='gpt-3.5-turbo',
messages=[
{'role': 'system', 'content': 'You are a intelligent assistant.'},
{'role': 'user', 'content': 'Return 3 different random JSON objects each has more than 500 characters.'},
],
stream=True
)
stream = Stream()
for chunk_obj in chat:
chunk_content = chunk_obj['choices'][0]['delta'].get('content', '')
for obj in stream.extract(chunk_content):
print(json.loads(obj))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
truncjson-1.2.5.tar.gz
(132.1 kB
view hashes)
Built Distributions
Close
Hashes for truncjson-1.2.5-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a8952999c9b8229b5f8fe8891e81d0c44aa6cee4e1e20ba22bd62c5c9bc205f9 |
|
MD5 | 0b85753fc8730dab7b8ff120022485e4 |
|
BLAKE2b-256 | 386ac7b758f54f4887c29efabeb21cf8da32bdee63ca4c2ba3a449a1f78ad1d6 |
Close
Hashes for truncjson-1.2.5-cp312-cp312-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a04b4e2166bad29901de2fe8443b5ce2e92048ef62d0cddec39c5bc9d8a912eb |
|
MD5 | 4403836d13a1bceab5d5685a9624c9fb |
|
BLAKE2b-256 | 4544c930191861faf6fdf118dec1f8685db69852242ca27acfe948b20eb143f4 |
Close
Hashes for truncjson-1.2.5-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b273cd6c4c26689f71bbd77e2aa08f2730dc3b6c71c4e9a5013c25b42e011e1f |
|
MD5 | f68074f896b1850d1bc2fefcf389c387 |
|
BLAKE2b-256 | 751d0d5ffb2e72960a60a5bfd1f447a5cea12a3bc5a8e95a06a176bbf9e12afc |