Truncated JSON completer
Project description
truncjson
A library to complete truncated JSON strings.
Use Cases
- When you possess a JSON file that is truncated, and you wish to parse its current content.
- When you are receiving JSON content as a stream from services like ChatGPT. This allows you to parse the JSON content each time you receive a new chunk.
Simple Usage
from truncjson import complete
result = complete('{"k1":"v1","k2":20.5,"k3":[12')
print(result)
# Output: {"k1":"v1","k2":20.5,"k3":[12]}
Stream Usage
Here we created a dummy stream with a list of chunks.
from truncjson import Stream
chunks = [
'Here yo',
'u go {"k1":"v1","k2":2',
'0.5,"k3":[12,34,"56",tr',
'ue,{"k31":"v31","k32":false,"k33":n',
'ull}]}\nAnd ano',
'ther one {"k',
'ey":1.',
'12} {"a',
'b":"cd"}'
]
stream = Stream()
for chunk in chunks:
print(stream.extract(chunk))
# Output:
# []
# ['{"k1":"v1","k2":2}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"k":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.0}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"a":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"ab":"cd"}']
ChatGPT Usage
import openai
import json
from truncjson import Stream
openai.api_key = '###'
chat = openai.ChatCompletion.create(
model='gpt-3.5-turbo',
messages=[
{'role': 'system', 'content': 'You are a intelligent assistant.'},
{'role': 'user', 'content': 'Return 3 different random JSON objects each has more than 500 characters.'},
],
stream=True
)
stream = Stream()
for chunk_obj in chat:
chunk_content = chunk_obj['choices'][0]['delta'].get('content', '')
for obj in stream.extract(chunk_content):
print(json.loads(obj))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
truncjson-1.2.4.tar.gz
(130.6 kB
view hashes)
Built Distributions
Close
Hashes for truncjson-1.2.4-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 491523ec1b0a6852bda46741fce4d42ca2e8d78cfbe9568612d12b9cbece7b87 |
|
MD5 | a09a76db7d2b74788f6ade010357de7d |
|
BLAKE2b-256 | cc48496c85e238659ec99a5e4bc1c1d5a003e2993ce3356eace0abde97f283d6 |
Close
Hashes for truncjson-1.2.4-cp311-cp311-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72e8c767be8bcf05437c6ed83eeefbe149c2c582a5fb96345971373e51a7ec03 |
|
MD5 | b1bcca951546e0b4bd22de10d5e7199e |
|
BLAKE2b-256 | 9f6b0757fa49f2ec4c868c8a4e4b91acc314076641ef61150dfe91765a99e767 |
Close
Hashes for truncjson-1.2.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9ef69de51ba3cfe2512bcc4149a82011932e43a512eaf0ee71c212f44e30baff |
|
MD5 | b99e050fd5ec5e2c397476835dc462f2 |
|
BLAKE2b-256 | c956dee3c334cdd6de2c048892823bb86c133154a44940dbab98e08db49851bd |