Skip to main content

A package to repair broken json strings

Project description

PyPI Python version PyPI downloads PyPI Downloads Github Sponsors GitHub Repo stars

English | 中文

This simple package can be used to fix an invalid json string. To know all cases in which this package will work, check out the unit test.

banner


Think about sponsoring this library!

This library is free for everyone and it's maintained and developed as a side project so, if you find this library useful for your work, consider becoming a sponsor via this link: https://github.com/sponsors/mangiucugna

Premium sponsors


Demo

If you are unsure if this library will fix your specific problem, or simply want your json validated online, you can visit the demo site on GitHub pages: https://mangiucugna.github.io/json_repair/

Or hear an audio deepdive generate by Google's NotebookLM for an introduction to the module


Motivation

Some LLMs are a bit iffy when it comes to returning well formed JSON data, sometimes they skip a parentheses and sometimes they add some words in it, because that's what an LLM does. Luckily, the mistakes LLMs make are simple enough to be fixed without destroying the content.

I searched for a lightweight python package that was able to reliably fix this problem but couldn't find any.

So I wrote one

Supported use cases

Fixing Syntax Errors in JSON

  • Missing quotes, misplaced commas, unescaped characters, and incomplete key-value pairs.
  • Missing quotation marks, improperly formatted values (true, false, null), and repairs corrupted key-value structures.

Repairing Malformed JSON Arrays and Objects

  • Incomplete or broken arrays/objects by adding necessary elements (e.g., commas, brackets) or default values (null, "").
  • The library can process JSON that includes extra non-JSON characters like comments or improperly placed characters, cleaning them up while maintaining valid structure.

Auto-Completion for Missing JSON Values

  • Automatically completes missing values in JSON fields with reasonable defaults (like empty strings or null), ensuring validity.

How to use

Install the library with pip

pip install json-repair

then you can use use it in your code like this

from json_repair import repair_json

good_json_string = repair_json(bad_json_string)
# If the string was super broken this will return an empty string

You can use this library to completely replace json.loads():

import json_repair

decoded_object = json_repair.loads(json_string)

or just

import json_repair

decoded_object = json_repair.repair_json(json_string, return_objects=True)

Avoid this antipattern

Some users of this library adopt the following pattern:

obj = {}
try:
    obj = json.loads(string)
except json.JSONDecodeError as e:
    obj = json_repair.loads(string)
    ...

This is wasteful because json_repair will already verify for you if the JSON is valid, if you still want to do that then add skip_json_loads=True to the call as explained the section below.

Read json from a file or file descriptor

JSON repair provides also a drop-in replacement for json.load():

import json_repair

try:
    file_descriptor = open(fname, 'rb')
except OSError:
    ...

with file_descriptor:
    decoded_object = json_repair.load(file_descriptor)

and another method to read from a file:

import json_repair

try:
    decoded_object = json_repair.from_file(json_file)
except OSError:
    ...
except IOError:
    ...

Keep in mind that the library will not catch any IO-related exception and those will need to be managed by you

Non-Latin characters

When working with non-Latin characters (such as Chinese, Japanese, or Korean), you need to pass ensure_ascii=False to repair_json() in order to preserve the non-Latin characters in the output.

Here's an example using Chinese characters:

repair_json("{'test_chinese_ascii':'统一码'}")

will return

{"test_chinese_ascii": "\u7edf\u4e00\u7801"}

Instead passing ensure_ascii=False:

repair_json("{'test_chinese_ascii':'统一码'}", ensure_ascii=False)

will return

{"test_chinese_ascii": "统一码"}

JSON dumps parameters

More in general, repair_json will accept all parameters that json.dumps accepts and just pass them through (for example indent)

Performance considerations

If you find this library too slow because is using json.loads() you can skip that by passing skip_json_loads=True to repair_json. Like:

from json_repair import repair_json

good_json_string = repair_json(bad_json_string, skip_json_loads=True)

I made a choice of not using any fast json library to avoid having any external dependency, so that anybody can use it regardless of their stack.

Some rules of thumb to use:

  • Setting return_objects=True will always be faster because the parser returns an object already and it doesn't have serialize that object to JSON
  • skip_json_loads is faster only if you 100% know that the string is not a valid JSON
  • If you are having issues with escaping pass the string as raw string like: r"string with escaping\""

Strict mode

By default json_repair does its best to “fix” input, even when the JSON is far from valid.
In some scenarios you want the opposite behavior and need the parser to error out instead of repairing; pass strict=True to repair_json, loads, load, or from_file to enable that mode:

from json_repair import repair_json

repair_json(bad_json_string, strict=True)

The CLI exposes the same behavior with json_repair --strict input.json (or piping data via stdin).

In strict mode the parser raises ValueError as soon as it encounters structural issues such as duplicate keys, missing : separators, empty keys/values introduced by stray commas, multiple top-level elements, or other ambiguous constructs. This is useful when you just need validation with friendlier error messages while still benefiting from json_repair’s resilience elsewhere in your stack.

Strict mode still honors skip_json_loads=True; combining them lets you skip the initial json.loads check but still enforce strict parsing rules.

Schema-guided repairs

Alpha feature (not yet in stable releases). Schema-guided repairs are currently shipped only in alpha builds (e.g., 0.56.0-alpha.*). The API and behavior may change or break between alpha releases.

You can guide repairs with a JSON Schema (or a Pydantic v2 model). When enabled, the parser will:

  • Fill missing values (defaults, required fields).
  • Coerce scalars where safe (e.g., "1"1 for integer fields).
  • Drop properties/items that the schema disallows.

This is especially useful when you need deterministic, schema-valid outputs for downstream validation, storage, or typed processing. If the input cannot be repaired to satisfy the schema, json_repair raises ValueError.

Install the optional dependencies:

pip install 'json-repair[schema]'

(For CLI usage, you can also use pipx install 'json-repair[schema]'.)

Schema guidance is skipped for already-valid JSON unless you pass skip_json_loads=True (this forces the parser to run even on valid JSON). Schema guidance is mutually exclusive with strict=True.

from json_repair import repair_json

schema = {
    "type": "object",
    "properties": {"value": {"type": "integer"}},
    "required": ["value"],
}

repair_json('{"value": "1"}', schema=schema, skip_json_loads=True, return_objects=True)

Pydantic v2 model example:

from pydantic import BaseModel, Field
from json_repair import repair_json


class Payload(BaseModel):
    value: int
    tags: list[str] = Field(default_factory=list)


repair_json(
    '{"value": "1", "tags": }',
    schema=Payload,
    skip_json_loads=True,
    return_objects=True,
)

Use json_repair with streaming

Sometimes you are streaming some data and want to repair the JSON coming from it. Normally this won't work but you can pass stream_stable to repair_json() or loads() to make it work:

stream_output = repair_json(stream_input, stream_stable=True)

Use json_repair from CLI

Install the library for command-line with:

pipx install json-repair

to know all options available:

$ json_repair -h
usage: json_repair [-h] [-i] [-o TARGET] [--ensure_ascii] [--indent INDENT]
                   [--skip-json-loads] [--schema SCHEMA] [--schema-model MODEL]
                   [--strict] [filename]

Repair and parse JSON files.

positional arguments:
  filename              The JSON file to repair (if omitted, reads from stdin)

options:
  -h, --help            show this help message and exit
  -i, --inline          Replace the file inline instead of returning the output to stdout
  -o TARGET, --output TARGET
                        If specified, the output will be written to TARGET filename instead of stdout
  --ensure_ascii        Pass ensure_ascii=True to json.dumps()
  --indent INDENT       Number of spaces for indentation (Default 2)
  --skip-json-loads     Skip initial json.loads validation (needed to force schema on valid JSON)
  --schema SCHEMA       Path to a JSON Schema file that guides repairs
  --schema-model MODEL  Pydantic v2 model in 'module:ClassName' form that guides repairs
  --strict              Raise on duplicate keys, missing separators, empty keys/values, and similar structural issues instead of repairing them

Adding to requirements

Please pin this library only on the major version!

We use TDD and strict semantic versioning, there will be frequent updates and no breaking changes in minor and patch versions. To ensure that you only pin the major version of this library in your requirements.txt, specify the package name followed by the major version and a wildcard for minor and patch versions. For example:

json_repair==0.*

In this example, any version that starts with 0. will be acceptable, allowing for updates on minor and patch versions.


How to cite

If you are using this library in your academic work (as I know many folks are) please find the BibTex here:

@software{Baccianella_JSON_Repair_-_2025,
    author  = "Stefano {Baccianella}",
    month   = "feb",
    title   = "JSON Repair - A python module to repair invalid JSON, commonly used to parse the output of LLMs",
    url     = "https://github.com/mangiucugna/json_repair",
    version = "0.39.1",
    year    = 2025
}

Thank you for citing my work and please send me a link to the paper if you can!


How it works

This module will parse the JSON file following the BNF definition:

<json> ::= <primitive> | <container>

<primitive> ::= <number> | <string> | <boolean>
; Where:
; <number> is a valid real number expressed in one of a number of given formats
; <string> is a string of valid characters enclosed in quotes
; <boolean> is one of the literal strings 'true', 'false', or 'null' (unquoted)

<container> ::= <object> | <array>
<array> ::= '[' [ <json> *(', ' <json>) ] ']' ; A sequence of JSON values separated by commas
<object> ::= '{' [ <member> *(', ' <member>) ] '}' ; A sequence of 'members'
<member> ::= <string> ': ' <json> ; A pair consisting of a name, and a JSON value

If something is wrong (a missing parentheses or quotes for example) it will use a few simple heuristics to fix the JSON string:

  • Add the missing parentheses if the parser believes that the array or object should be closed
  • Quote strings or add missing single quotes
  • Adjust whitespaces and remove line breaks

I am sure some corner cases will be missing, if you have examples please open an issue or even better push a PR

Contributing

If you want to contribute, start with CONTRIBUTING.md and read the Code Wiki writeup for a tour of the codebase and key entry points: https://codewiki.google/github.com/mangiucugna/json_repair

How to develop

Use uv to set up the dev environment and run tooling:

uv sync --group dev
uv run pre-commit run --all-files
uv run pytest

Make sure that the Github Actions running after pushing a new commit don't fail as well.

How to release

You will need owner access to this repository

  • Edit pyproject.toml and update the version number appropriately using semver notation
  • Commit and push all changes to the repository before continuing or the next steps will fail
  • Run python -m build
  • Create a new release in Github, making sure to tag all the issues solved and contributors. Create the new tag, same as the one in the build configuration
  • Once the release is created, a new Github Actions workflow will start to publish on Pypi, make sure it didn't fail

Repair JSON in other programming languages


Star History

Star History Chart

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

json_repair-0.56.0.tar.gz (53.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

json_repair-0.56.0-py3-none-any.whl (38.7 kB view details)

Uploaded Python 3

File details

Details for the file json_repair-0.56.0.tar.gz.

File metadata

  • Download URL: json_repair-0.56.0.tar.gz
  • Upload date:
  • Size: 53.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for json_repair-0.56.0.tar.gz
Algorithm Hash digest
SHA256 d6159e9b32a8a310a7e016862a255c4ce19221417f80847a5be146f0cca753a7
MD5 763581c3e8294936f5ddc33907e77a3d
BLAKE2b-256 2e9013e25364ffad227896a1853fb05bf26af6186d97dc3c30ed7317c2edf33a

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_repair-0.56.0.tar.gz:

Publisher: python-publish.yml on mangiucugna/json_repair

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file json_repair-0.56.0-py3-none-any.whl.

File metadata

  • Download URL: json_repair-0.56.0-py3-none-any.whl
  • Upload date:
  • Size: 38.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for json_repair-0.56.0-py3-none-any.whl
Algorithm Hash digest
SHA256 41005b8c1d4ca74dcb902bab6049467d4feb5e18e1d1621b6ca73c48dae04798
MD5 c9e7e4bcd2ad26aff59e10934ccabacd
BLAKE2b-256 584faa233e9849c521e7480c02e253d1e1c9c58d9ad5e9350a702650b797d3a5

See more details on using hashes here.

Provenance

The following attestation bundles were made for json_repair-0.56.0-py3-none-any.whl:

Publisher: python-publish.yml on mangiucugna/json_repair

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page