pyjson-toolkit 1.0.0

A comprehensive Python toolkit for loading, validating, searching, flattening, merging, comparing, transforming, and converting JSON data.

pyjson-toolkit

A comprehensive Python toolkit for loading, transforming, validating, comparing, searching, formatting, and converting JSON data.

---

Why pyjson-toolkit?

Working with JSON often requires installing multiple libraries:

Task	Common Solution
Read & Write JSON	json
Validate JSON	jsonschema
Convert YAML	PyYAML
Convert XML	xmltodict
Flatten Objects	Various utility packages
Compare JSON	Custom scripts
Mask Sensitive Data	Custom implementations

pyjson-toolkit combines these capabilities into a single, consistent, and easy-to-use API.

---

Key Features

JSON Processing

• Load JSON from files
• Save JSON to files
• Parse JSON strings
• Serialize Python objects
• Pretty-print JSON
• Minify JSON

Data Transformation

• Flatten nested JSON structures
• Restore flattened JSON
• Deep merge JSON objects
• Recursive sorting
• Rename keys
• Remove keys
• Remove null values
• Remove empty values

Search

• Dot notation support
• Nested object traversal
• Array indexing
• Wildcard search
• Key existence checks

Validation

• JSON Schema validation
• Required field validation
• Missing key detection
• Automatic schema generation

Comparison

• JSON diff
• Deep object comparison
• API response comparison

Security

• Mask sensitive fields
• Remove confidential data
• Configurable masking rules

Conversion

• CSV ↔ JSON
• XML ↔ JSON
• YAML ↔ JSON

Command Line Interface

• Pretty print JSON
• Flatten JSON
• Merge files
• Validate JSON
• Compare JSON
• Convert between formats

---

Installation

Requirements

• Python 3.10 or newer

Install from PyPI

pip install pyjson-toolkit

Verify Installation

from pyjson_toolkit import JsonTools

print(JsonTools.__version__)

---

Quick Start

Load a JSON file.

from pyjson_toolkit import JsonTools

data = JsonTools.load("sample.json")

Save a JSON file.

JsonTools.save(data, "output.json")

Parse a JSON string.

json_string = '{"name":"Alice","age":25}'

data = JsonTools.loads(json_string)

Serialize an object.

text = JsonTools.dumps(data)

Pretty-print JSON.

formatted = JsonTools.pretty(data)

Minify JSON.

compact = JsonTools.minify(data)

---

Basic Example

from pyjson_toolkit import JsonTools

users = JsonTools.load("users.json")

users = JsonTools.remove_nulls(users)

users = JsonTools.sort_recursively(users)

JsonTools.save(users, "clean_users.json")

---

Supported Data Types

Type	Supported
dict	Yes
list	Yes
tuple	Converted to list
str	Yes
int	Yes
float	Yes
bool	Yes
None	Yes

---

Package Overview

pyjson-toolkit/
│
├── pyjson_toolkit/
│   ├── core.py
│   ├── io.py
│   ├── flatten.py
│   ├── merge.py
│   ├── diff.py
│   ├── search.py
│   ├── validation.py
│   ├── schema.py
│   ├── converters.py
│   ├── masking.py
│   ├── cleaning.py
│   ├── sorting.py
│   ├── compare.py
│   ├── cli.py
│   └── utils.py
│
├── tests/
├── docs/
├── examples/
└── benchmarks/

---

Core API

The library exposes a single entry point.

from pyjson_toolkit import JsonTools

Every major feature is available through this class, providing a consistent and discoverable API.

The following sections describe each feature in detail.

---

JSON Transformation

pyjson-toolkit provides a rich set of utilities for transforming JSON structures without modifying the original data unless explicitly requested.

Flatten JSON

Flatten nested JSON objects into a single-level dictionary using dot notation.

Input

data = {
    "user": {
        "name": "Alice",
        "address": {
            "city": "London",
            "country": "UK"
        }
    }
}

Usage

flat = JsonTools.flatten(data)

Output

{
    "user.name": "Alice",
    "user.address.city": "London",
    "user.address.country": "UK"
}

---

Flatten Options

flat = JsonTools.flatten(
    data,
    separator=".",
    preserve_lists=True
)

Parameters

Parameter	Description
separator	Key separator
preserve_lists	Preserve list indices

---

Unflatten JSON

Convert flattened JSON back into its original nested structure.

Usage

nested = JsonTools.unflatten(flat)

---

Merge JSON Objects

Merge two or more JSON documents recursively.

Example

config_a = {
    "database": {
        "host": "localhost",
        "port": 3306
    }
}

config_b = {
    "database": {
        "user": "admin",
        "password": "secret"
    }
}

merged = JsonTools.merge(config_a, config_b)

Output

{
    "database": {
        "host": "localhost",
        "port": 3306,
        "user": "admin",
        "password": "secret"
    }
}

---

Merge Strategies

The merge operation supports multiple conflict-resolution strategies.

merged = JsonTools.merge(
    left,
    right,
    strategy="overwrite"
)

Supported strategies

Strategy	Description
overwrite	Replace existing values
keep	Preserve original values
append	Append list values
recursive	Merge nested dictionaries

---

Searching JSON

Locate values using an intuitive dot notation syntax.

Nested Object

value = JsonTools.search(
    data,
    "customer.address.city"
)

---

Array Index

price = JsonTools.search(
    data,
    "orders[0].price"
)

---

Wildcards

items = JsonTools.search(
    data,
    "orders[*].id"
)

---

Default Value

country = JsonTools.search(
    data,
    "customer.country",
    default="Unknown"
)

---

Renaming Keys

Rename one or more keys recursively.

updated = JsonTools.rename_key(
    data,
    "custId",
    "customerId"
)

Rename multiple keys.

updated = JsonTools.rename_keys(
    data,
    {
        "custId": "customerId",
        "addr": "address",
        "tel": "phone"
    }
)

---

Removing Keys

Remove unwanted keys recursively.

clean = JsonTools.remove_keys(
    data,
    [
        "password",
        "token",
        "secret"
    ]
)

---

Cleaning JSON

Remove unnecessary values before storage or transmission.

Remove Null Values

clean = JsonTools.remove_nulls(data)

Example

Input

{
    "name": "Alice",
    "email": None,
    "phone": None
}

Output

{
    "name": "Alice"
}

---

Remove Empty Values

clean = JsonTools.remove_empty(data)

Removes

• Empty strings
• Empty dictionaries
• Empty lists
• Empty tuples

---

Remove Duplicate Objects

clean = JsonTools.remove_duplicates(data)

---

Sorting

Sort Top-Level Keys

sorted_json = JsonTools.sort_keys(data)

---

Recursive Sorting

sorted_json = JsonTools.sort_recursively(data)

This recursively sorts every nested dictionary in the document.

---

Filtering

Select only specific fields.

filtered = JsonTools.select_keys(
    data,
    [
        "id",
        "name",
        "email"
    ]
)

Exclude fields.

filtered = JsonTools.exclude_keys(
    data,
    [
        "password",
        "token"
    ]
)

---

Mask Sensitive Information

Mask confidential information before logging or sharing.

masked = JsonTools.mask(
    data,
    [
        "password",
        "secret",
        "token",
        "apiKey",
        "creditCard"
    ]
)

Output

{
    "username": "admin",
    "password": "********",
    "token": "********"
}

Custom mask character.

masked = JsonTools.mask(
    data,
    fields=["password"],
    mask="X"
)

Output

{
    "password": "XXXXXXXX"
}

---

Working with Large Files

For large JSON documents, use streaming methods.

for record in JsonTools.stream("large.json"):
    print(record)

Streaming minimizes memory usage and is suitable for processing multi-gigabyte JSON files.

---

Validation

Reliable data validation is essential when working with APIs, configuration files, ETL pipelines, and data processing workflows.

pyjson-toolkit provides built-in validation utilities powered by JSON Schema while also offering lightweight validation helpers.

---

Validate Against a JSON Schema

Validate a JSON document using a JSON Schema.

from pyjson_toolkit import JsonTools

schema = JsonTools.load("schema.json")
data = JsonTools.load("employee.json")

JsonTools.validate(schema, data)

If validation fails, a descriptive exception is raised.

---

Validate from Files

JsonTools.validate_file(
    "schema.json",
    "employee.json"
)

---

Validation Result

Instead of raising exceptions, return a validation result.

result = JsonTools.validate(
    schema,
    data,
    raise_exception=False
)

print(result.valid)
print(result.errors)

Example

ValidationResult(
    valid=False,
    errors=[
        "$.employee.age is required",
        "$.department.id must be integer"
    ]
)

---

Validate Required Fields

Ensure required fields exist.

JsonTools.validate_required_fields(
    data,
    [
        "id",
        "name",
        "email"
    ]
)

Nested fields are supported.

JsonTools.validate_required_fields(
    data,
    [
        "customer.id",
        "customer.address.city",
        "orders"
    ]
)

---

Find Missing Keys

Compare a JSON document against expected fields.

missing = JsonTools.find_missing_keys(
    data,
    [
        "customer.id",
        "customer.name",
        "customer.phone"
    ]
)

Output

[
    "customer.phone"
]

---

Schema Generation

Generate a JSON Schema from an existing JSON document.

schema = JsonTools.generate_schema(data)

Example

Input

{
    "id": 1,
    "name": "Alice",
    "active": True
}

Generated Schema

{
    "type": "object",
    "properties": {
        "id": {
            "type": "integer"
        },
        "name": {
            "type": "string"
        },
        "active": {
            "type": "boolean"
        }
    },
    "required": [
        "id",
        "name",
        "active"
    ]
}

---

Infer Data Types

Determine the structure of any JSON document.

types = JsonTools.infer_types(data)

Output

{
    "id": "integer",
    "name": "string",
    "salary": "number",
    "active": "boolean"
}

---

Compare JSON Documents

Compare two JSON documents.

diff = JsonTools.diff(old_data, new_data)

Example Output

{
    "added": [
        "address.city"
    ],
    "removed": [
        "phone"
    ],
    "modified": [
        "salary"
    ]
}

---

Deep Comparison

result = JsonTools.compare(
    expected,
    actual
)

Returns

ComparisonResult(
    equal=True,
    differences=[]
)

or

ComparisonResult(
    equal=False,
    differences=[
        "$.salary differs",
        "$.address.city missing"
    ]
)

---

Compare API Responses

Designed specifically for automated API testing.

result = JsonTools.compare_api_response(
    expected,
    actual
)

Supports

• Nested comparison
• Missing fields
• Additional fields
• Type mismatch detection
• Value mismatch detection

---

Ignore Specific Fields

Ignore dynamic values during comparison.

JsonTools.compare_api_response(
    expected,
    actual,
    ignore=[
        "timestamp",
        "requestId",
        "createdAt",
        "updatedAt"
    ]
)

---

Tolerance for Numeric Values

Useful for scientific calculations.

JsonTools.compare(
    expected,
    actual,
    tolerance=0.001
)

---

Compare Arrays

Order-sensitive comparison.

JsonTools.compare(
    expected,
    actual,
    ignore_order=False
)

Order-insensitive comparison.

JsonTools.compare(
    expected,
    actual,
    ignore_order=True
)

---

Generate Difference Report

Produce a human-readable report.

report = JsonTools.diff_report(
    expected,
    actual
)

print(report)

Example

Difference Report
=================

Modified
--------

customer.name

Expected:
Alice

Actual:
Bob

Removed
-------

phone

Added
-----

address.country

---

Export Difference Report

JsonTools.export_diff(
    expected,
    actual,
    "report.json"
)

or

JsonTools.export_diff(
    expected,
    actual,
    "report.html"
)

---

Exception Handling

All library exceptions inherit from a common base class.

from pyjson_toolkit import JsonToolkitError

try:
    JsonTools.validate(schema, data)

except JsonToolkitError as exc:
    print(exc)

Specialized exceptions include:

• InvalidJsonError
• JsonValidationError
• JsonSchemaError
• JsonConversionError
• JsonComparisonError
• JsonSearchError
• JsonFileError

---

Logging

Enable verbose logging.

JsonTools.enable_logging(level="INFO")

Disable logging.

JsonTools.disable_logging()

The toolkit integrates with Python's standard logging module and supports custom loggers.

---

Format Conversion

pyjson-toolkit provides built-in converters for working with multiple data formats commonly used in APIs, configuration management, reporting, and data exchange.

All conversion methods are designed to preserve the original data structure whenever possible.

---

CSV to JSON

Convert CSV files into JSON.

from pyjson_toolkit import JsonTools

data = JsonTools.csv_to_json("employees.csv")

Example CSV

id,name,department,salary
1,Alice,Engineering,75000
2,Bob,Sales,65000

Output

[
    {
        "id": 1,
        "name": "Alice",
        "department": "Engineering",
        "salary": 75000
    },
    {
        "id": 2,
        "name": "Bob",
        "department": "Sales",
        "salary": 65000
    }
]

---

JSON to CSV

Convert JSON arrays into CSV files.

JsonTools.json_to_csv(
    data,
    "employees.csv"
)

Supported features

• Automatic header generation
• Custom delimiter
• UTF-8 encoding
• Nested object flattening
• Optional quoting

---

XML to JSON

Convert XML documents into Python dictionaries.

data = JsonTools.xml_to_json(
    "sample.xml"
)

Example XML

<employee>
    <id>1</id>
    <name>Alice</name>
</employee>

Output

{
    "employee": {
        "id": "1",
        "name": "Alice"
    }
}

---

JSON to XML

Convert JSON into XML.

xml = JsonTools.json_to_xml(data)

Save directly to a file.

JsonTools.json_to_xml(
    data,
    output_file="employee.xml"
)

---

YAML to JSON

Convert YAML configuration files.

data = JsonTools.yaml_to_json(
    "config.yaml"
)

Example

database:
  host: localhost
  port: 5432

Output

{
    "database": {
        "host": "localhost",
        "port": 5432
    }
}

---

JSON to YAML

yaml_text = JsonTools.json_to_yaml(data)

Save directly.

JsonTools.json_to_yaml(
    data,
    output_file="config.yaml"
)

---

Conversion Matrix

From	To	Supported
JSON	JSON	Yes
JSON	CSV	Yes
CSV	JSON	Yes
JSON	XML	Yes
XML	JSON	Yes
JSON	YAML	Yes
YAML	JSON	Yes

---

Batch Processing

Process multiple files with a single command.

JsonTools.batch_convert(
    source_directory="input",
    destination_directory="output",
    source_format="json",
    destination_format="yaml"
)

---

File Utilities

Check whether a file contains valid JSON.

JsonTools.is_json_file(
    "sample.json"
)

Validate a JSON file.

JsonTools.validate_json_file(
    "sample.json"
)

Read JSON safely.

data = JsonTools.safe_load(
    "config.json"
)

Write JSON safely.

JsonTools.safe_save(
    data,
    "output.json"
)

---

Command Line Interface

The package includes a command-line application named pyjson.

Display help.

pyjson --help

Display version.

pyjson --version

---

Pretty Print

pyjson pretty users.json

---

Minify

pyjson minify users.json

---

Flatten

pyjson flatten users.json

---

Unflatten

pyjson unflatten users.json

---

Merge

pyjson merge config1.json config2.json

---

Compare

pyjson compare expected.json actual.json

---

Validate

pyjson validate schema.json data.json

---

Generate Schema

pyjson schema employee.json

---

CSV Conversion

pyjson csv2json employees.csv

pyjson json2csv employees.json

---

XML Conversion

pyjson xml2json sample.xml

pyjson json2xml sample.json

---

YAML Conversion

pyjson yaml2json config.yaml

pyjson json2yaml config.json

---

Performance

The toolkit is optimized for handling both small configuration files and large datasets.

Highlights include:

• Recursive algorithms optimized for nested structures
• Efficient dictionary traversal
• Streaming support for large JSON documents
• Minimal memory allocations
• Type-safe implementations
• Comprehensive error handling

Performance benchmarks are included in the benchmarks/ directory.

---

Type Hints

The entire public API includes complete type hints.

Example

from typing import Any

data: dict[str, Any] = JsonTools.load("sample.json")

This enables:

• IDE autocompletion
• Static analysis
• MyPy compatibility
• Better developer experience

---

Thread Safety

All transformation operations are designed to avoid mutating the original input unless explicitly requested.

This makes the toolkit suitable for:

• Multi-threaded applications
• Web APIs
• Background workers
• Data pipelines

---

Supported Python Versions

Python Version	Supported
3.10	Yes
3.11	Yes
3.12	Yes
3.13	Yes

---

Supported Platforms

• Windows
• Linux
• macOS

The toolkit is platform-independent and relies only on pure Python dependencies.

---

API Reference

The JsonTools class exposes a unified API for all supported operations.

Category	Method
JSON I/O	load()
JSON I/O	save()
JSON I/O	loads()
JSON I/O	dumps()
Formatting	pretty()
Formatting	minify()
Formatting	sort_keys()
Formatting	sort_recursively()
Structure	flatten()
Structure	unflatten()
Structure	merge()
Structure	rename_key()
Structure	rename_keys()
Structure	remove_keys()
Structure	remove_nulls()
Structure	remove_empty()
Structure	remove_duplicates()
Structure	select_keys()
Structure	exclude_keys()
Search	search()
Search	exists()
Validation	validate()
Validation	validate_file()
Validation	generate_schema()
Validation	validate_required_fields()
Validation	find_missing_keys()
Validation	infer_types()
Comparison	compare()
Comparison	diff()
Comparison	diff_report()
Comparison	export_diff()
Comparison	compare_api_response()
Security	mask()
Conversion	csv_to_json()
Conversion	json_to_csv()
Conversion	xml_to_json()
Conversion	json_to_xml()
Conversion	yaml_to_json()
Conversion	json_to_yaml()
Utilities	stream()
Utilities	safe_load()
Utilities	safe_save()
Utilities	enable_logging()
Utilities	disable_logging()

---

Examples

The repository includes practical examples demonstrating the most common use cases.

examples/
├── basic_usage.py
├── flatten_example.py
├── merge_example.py
├── validation_example.py
├── comparison_example.py
├── masking_example.py
├── conversion_example.py
└── cli_examples.md

Each example is self-contained and can be executed independently.

---

Documentation

The complete documentation includes:

• Installation Guide
• User Guide
• API Reference
• CLI Reference
• Examples
• Migration Guide
• Frequently Asked Questions
• Release Notes

Documentation can be generated locally using MkDocs.

mkdocs serve

Build the static documentation site.

mkdocs build

---

Testing

Run the full test suite.

pytest

Generate a coverage report.

pytest --cov=pyjson_toolkit

Run a specific test file.

pytest tests/test_flatten.py

Run a single test.

pytest tests/test_merge.py::test_recursive_merge

---

Code Quality

Format the project.

black .

Run the linter.

ruff check .

Run static type checking.

mypy pyjson_toolkit

---

Project Structure

pyjson-toolkit/
│
├── pyjson_toolkit/
│   ├── __init__.py
│   ├── core.py
│   ├── io.py
│   ├── flatten.py
│   ├── merge.py
│   ├── search.py
│   ├── validation.py
│   ├── schema.py
│   ├── diff.py
│   ├── compare.py
│   ├── masking.py
│   ├── converters.py
│   ├── cleaning.py
│   ├── sorting.py
│   ├── cli.py
│   ├── utils.py
│   ├── constants.py
│   ├── exceptions.py
│   └── types.py
│
├── tests/
├── docs/
├── examples/
├── benchmarks/
├── scripts/
└── .github/

---

Frequently Asked Questions

Why use pyjson-toolkit instead of Python's built-in json module?

The built-in json module focuses on serialization and deserialization. pyjson-toolkit extends those capabilities with validation, transformation, comparison, schema generation, search, masking, and data format conversion.

---

Does the toolkit modify the original object?

No. Methods return new objects unless an inplace=True option is explicitly provided.

---

Can I process very large JSON files?

Yes. Streaming APIs are available to reduce memory usage when processing large datasets.

---

Does it support JSON Schema?

Yes. Validation is implemented using the JSON Schema specification.

---

Is the project suitable for production?

Yes. The library is designed for production environments, with comprehensive tests, type hints, and cross-platform support.

---

Versioning

This project follows Semantic Versioning.

MAJOR.MINOR.PATCH

Examples

1.0.0
1.2.0
1.2.5
2.0.0

---

Changelog

See CHANGELOG.md for a complete history of releases.

---

Contributing

Contributions are welcome.

You can contribute by:

• Reporting bugs
• Suggesting new features
• Improving documentation
• Writing tests
• Optimizing performance
• Submitting pull requests

Please read CONTRIBUTING.md before contributing.

---

Security

If you discover a security vulnerability, please report it privately rather than opening a public issue.

Responsible disclosure helps protect users while fixes are being prepared.

---

License

This project is licensed under the MIT License.

See the LICENSE file for the full license text.

---

Author

Surender Elangovan

Email: [e-mail]

GitHub: https://github.com/SurenderElangovan

Repository:

https://github.com/SurenderElangovan/pyjson-toolkit

---

Acknowledgements

This project builds upon the excellent work of the Python open-source community, including:

• Python Standard Library
• jsonschema
• PyYAML
• xmltodict
• openpyxl
• pytest
• Black
• Ruff
• MyPy

---

Support

If you find this project useful:

• Star the GitHub repository.
• Report bugs or request features through GitHub Issues.
• Contribute improvements through pull requests.
• Share the project with the Python community.

Feedback and contributions are always welcome.

---

Citation

If you use pyjson-toolkit in your project, publication, or research, please cite the repository.

Surender Elangovan.
pyjson-toolkit: A comprehensive Python toolkit for JSON manipulation,
validation, comparison, transformation, and format conversion.
https://github.com/SurenderElangovan/pyjson-toolkit