Generate mock data using json schema supplied.
Project description
Data Generator - WIP
Overview
During every data project I came across a very basic problem where we have to wait for the test data. For fewer columns it's quite easy to generate the data using online utilities but those has certain limitations on number of columns and rows. Solve this, I invested some time to build a utility to generate the mock data based on the supplied json schema. This utility is using Python Faker module to randomly generate the test data.
How to use
Follow below step to run the utility. I am open to the suggestions, please add comment or mail me your suggestions.
Inputs
- input_json_schema_path: Provide absolute path of the json schema file. The sample json schema file should be
{
"type": "object",
"properties": {
"price": { "type": "number" },
"name": { "type": "string" },
"a": { "type": "integer" },
"b": { "type": "float" },
"c": { "type": "boolean" },
"dt": { "type": "date" },
"ts": { "type": "timestamp" },
"e": { "type": "boolean" }
}
}
-
output_file_format: The output file format should be one of the
"CSV","JSON","XML","EXCEL","PARQUET","ORC"
-
output_path: Absolute path where the generated output should be stored
-
number_of_rows: Number of output rows to be generated
-
Supported data types are:
"STRING","INT","INTEGER","NUMBER","FLOAT","DATE","BOOLEAN","BOOL","TIMESTAMP"
Pre-requisites
- Python 3.11.3
- Poetry 1.3.2
Steps to execute the utility
clone the repo
cd mock_data_generator
poetry install
- Sample command:
poetry run generate --input_json_schema_path=resources/schema.json --output_file_format=csv --output_path=output_data --number_of_rows=10
: If output path does not exists, it will create it and store the data inside the directory into data.csv file
Licensing
Distributed under the MIT license. See LICENSE
for more information.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mock_data_generator-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | f919fdd3697f6afb6982f3e823acb97143cd7d9ff58f1317c32787bba3be142a |
|
MD5 | 654117612a58e629d05f41677275859c |
|
BLAKE2b-256 | 4400f3692845718eb12037f716c5f7dfc4413bb72e2eff174c7e5e0e51df3a6a |
Hashes for mock_data_generator-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | df20bce86383d973a2982327b876d50b0ffa25d1c43548a26ee611e6c553c142 |
|
MD5 | bdb7404630cb4bf77c91895733137163 |
|
BLAKE2b-256 | be6452c851362bf43dca209fba38cf7de2ab4dcda8f482232662486435f182c4 |