Truncated JSON completer
Project description
truncjson
A library to complete truncated JSON strings.
Use Cases
- When you possess a JSON file that is truncated, and you wish to parse its current content.
- When you are receiving JSON content as a stream from services like ChatGPT. This allows you to parse the JSON content each time you receive a new chunk.
Simple Usage
from truncjson import complete
result = complete('{"k1":"v1","k2":20.5,"k3":[12')
print(result)
# Output: {"k1":"v1","k2":20.5,"k3":[12]}
Stream Usage
Here we created a dummy stream with a list of chunks.
from truncjson import Stream
chunks = [
'Here yo',
'u go {"k1":"v1","k2":2',
'0.5,"k3":[12,34,"56",tr',
'ue,{"k31":"v31","k32":false,"k33":n',
'ull}]}\nAnd ano',
'ther one {"k',
'ey":1.',
'12} {"a',
'b":"cd"}'
]
stream = Stream()
for chunk in chunks:
print(stream.extract(chunk))
# Output:
# []
# ['{"k1":"v1","k2":2}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"k":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.0}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"a":null}']
# ['{"k1":"v1","k2":20.5,"k3":[12,34,"56",true,{"k31":"v31","k32":false,"k33":null}]}', '{"key":1.12}', '{"ab":"cd"}']
ChatGPT Usage
import openai
import json
from truncjson import Stream
openai.api_key = '###'
chat = openai.ChatCompletion.create(
model='gpt-3.5-turbo',
messages=[
{'role': 'system', 'content': 'You are a intelligent assistant.'},
{'role': 'user', 'content': 'Return 3 different random JSON objects each has more than 500 characters.'},
],
stream=True
)
stream = Stream()
for chunk_obj in chat:
chunk_content = chunk_obj['choices'][0]['delta'].get('content', '')
for obj in stream.extract(chunk_content):
print(json.loads(obj))
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
truncjson-1.2.3.tar.gz
(130.7 kB
view hashes)
Built Distributions
Close
Hashes for truncjson-1.2.3-cp312-cp312-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ff9175717c74426d2395073995bdb90cb3273d77cb8364e3f2ed205562aeb498 |
|
MD5 | 77a66feb17cce894b4aae458d3f6312a |
|
BLAKE2b-256 | 885cc87f8d8422cb62be703358cb2636992a64ab46f1d10e2de8b79a6b2b89a4 |
Close
Hashes for truncjson-1.2.3-cp311-cp311-macosx_10_9_universal2.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 57f4c85783b0d1ca948a23d6142cea205e79669cb9719e4a77ab603e8fe6f409 |
|
MD5 | 6495bd4f985f012a4d17333aa913fafb |
|
BLAKE2b-256 | df682302cd6827230d6c53a5089e571eca49c56aec82953f8d96b478275baea5 |