String token parser
Project description
Token parser
More often than not I find myself creating scripts to generate test data, and those scripts always have some kind of token parser. So I decided to create this script in a more re-usable form.
What's the use-case for this?
Usually I create a json template and then generate a batch of test data. The json will act as the body of my test items. While this script will provide randomized data for each instance.
Installation
pip install token-parser
You can check out this project at PyPi.
What can this parse_token function do?
- Get current time
- Get current time (UTC)
- Convert a timestamp to datetime
- Convert a timestamp to datetime (UTC)
- Manipulate a datetime string and generate a datetime object
- Convert string to int
- Convert string to float
- Generate a random int
- Generate a random float
- Chose randomly an int from a list
- Chose randomly a float from a list
- Generate sequential int (positive and negative)
- Generate incremental (by N) int (positive and negative)
- Generate uuid4 strings (unique for whole session or individual)
Example
Here's an example. More in the examples folder
Consider the following JSON template:
{
"id": "$inc()",
"testSession": "$guid(true)",
"name": "Dolly",
"age": "$int(18, 42)",
"score": "$float(150, 9999)",
"status": "$int(1,2,3,4,5)",
"generationDate": "$utcNow()"
}
from token_parser.parsers import parse_token
# Helper variables with ISO8601 datetime format (utc and local)
ISO8601_DATE_FORMAT_UTC = "%Y-%m-%dT%H:%M:%S.%fZ"
ISO8601_DATE_FORMAT_LOCAL = ISO8601_DATE_FORMAT_UTC[:-1]
# Storing test items here
test_items = []
# Converting the string JSON to a dictionary (You could use json.load and get it from a file)
template = ast.literal_eval(JSON_TEMPLATE)
# Getting a initial datetime
test_data_creation_start = datetime.utcnow()
# Each datetime will be incremented by 15 minutes
created_at_delay = {"minutes": 15}
# I will create 50 test items
for i in range(50):
# Current test item
test_item = {}
for key, item in template.items():
# Parsing each token and adding it to the current item.
test_item[key] = parse_token(item)
# Adding an extra key named 'createdAt', which will be the initial date + 15 minutes
test_item["createdAt"] = parse_token(
f"$dateAdd({test_data_creation_start.strftime(ISO8601_DATE_FORMAT_UTC)}, {created_at_delay})")
# overwriting the initial datetime, so it will be incremented each time
test_data_creation_start = test_item["createdAt"]
# adding test item to the list
test_items.append(test_item)
# printing output
pprint(test_items)
The output for this would be:
[
{'age': 33,
'createdAt': datetime.datetime(2021, 7, 1, 1, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 183365, tzinfo=<UTC>),
'id': 1,
'name': 'Dolly',
'score': 9631.926551796414,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 35,
'createdAt': datetime.datetime(2021, 7, 1, 1, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 2,
'name': 'Dolly',
'score': 5486.760170377791,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 36,
'createdAt': datetime.datetime(2021, 7, 1, 1, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 3,
'name': 'Dolly',
'score': 383.9861640547723,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 28,
'createdAt': datetime.datetime(2021, 7, 1, 2, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 4,
'name': 'Dolly',
'score': 6644.5243644456095,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 27,
'createdAt': datetime.datetime(2021, 7, 1, 2, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 5,
'name': 'Dolly',
'score': 7275.414535471738,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 23,
'createdAt': datetime.datetime(2021, 7, 1, 2, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 6,
'name': 'Dolly',
'score': 4595.313386249013,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 35,
'createdAt': datetime.datetime(2021, 7, 1, 2, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 7,
'name': 'Dolly',
'score': 2265.5076487765696,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 29,
'createdAt': datetime.datetime(2021, 7, 1, 3, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 8,
'name': 'Dolly',
'score': 8333.836562119764,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 32,
'createdAt': datetime.datetime(2021, 7, 1, 3, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 9,
'name': 'Dolly',
'score': 7969.702985007152,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 41,
'createdAt': datetime.datetime(2021, 7, 1, 3, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 186086, tzinfo=<UTC>),
'id': 10,
'name': 'Dolly',
'score': 616.255951928468,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 38,
'createdAt': datetime.datetime(2021, 7, 1, 3, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 11,
'name': 'Dolly',
'score': 2124.78831535946,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 23,
'createdAt': datetime.datetime(2021, 7, 1, 4, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 12,
'name': 'Dolly',
'score': 1263.9157316326682,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 22,
'createdAt': datetime.datetime(2021, 7, 1, 4, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 13,
'name': 'Dolly',
'score': 6886.735759138181,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 40,
'createdAt': datetime.datetime(2021, 7, 1, 4, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 14,
'name': 'Dolly',
'score': 1440.5660523652627,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 35,
'createdAt': datetime.datetime(2021, 7, 1, 4, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 15,
'name': 'Dolly',
'score': 2022.2222952108284,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 23,
'createdAt': datetime.datetime(2021, 7, 1, 5, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 16,
'name': 'Dolly',
'score': 6974.875406639923,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 38,
'createdAt': datetime.datetime(2021, 7, 1, 5, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 17,
'name': 'Dolly',
'score': 3989.4146288683824,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 24,
'createdAt': datetime.datetime(2021, 7, 1, 5, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 18,
'name': 'Dolly',
'score': 7042.729058439558,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 28,
'createdAt': datetime.datetime(2021, 7, 1, 5, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 19,
'name': 'Dolly',
'score': 6516.063569457256,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 23,
'createdAt': datetime.datetime(2021, 7, 1, 6, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 187086, tzinfo=<UTC>),
'id': 20,
'name': 'Dolly',
'score': 8035.498749541888,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 29,
'createdAt': datetime.datetime(2021, 7, 1, 6, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188085, tzinfo=<UTC>),
'id': 21,
'name': 'Dolly',
'score': 5074.794512709819,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 41,
'createdAt': datetime.datetime(2021, 7, 1, 6, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188085, tzinfo=<UTC>),
'id': 22,
'name': 'Dolly',
'score': 6253.31611671207,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 42,
'createdAt': datetime.datetime(2021, 7, 1, 6, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188085, tzinfo=<UTC>),
'id': 23,
'name': 'Dolly',
'score': 864.6435456561902,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 26,
'createdAt': datetime.datetime(2021, 7, 1, 7, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188085, tzinfo=<UTC>),
'id': 24,
'name': 'Dolly',
'score': 9744.07103989777,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 34,
'createdAt': datetime.datetime(2021, 7, 1, 7, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188085, tzinfo=<UTC>),
'id': 25,
'name': 'Dolly',
'score': 1307.0134699144637,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 25,
'createdAt': datetime.datetime(2021, 7, 1, 7, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 26,
'name': 'Dolly',
'score': 9226.487429236373,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 20,
'createdAt': datetime.datetime(2021, 7, 1, 7, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 27,
'name': 'Dolly',
'score': 7230.284314784989,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 22,
'createdAt': datetime.datetime(2021, 7, 1, 8, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 28,
'name': 'Dolly',
'score': 9976.667139920644,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 28,
'createdAt': datetime.datetime(2021, 7, 1, 8, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 29,
'name': 'Dolly',
'score': 2336.5382817124732,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 32,
'createdAt': datetime.datetime(2021, 7, 1, 8, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 30,
'name': 'Dolly',
'score': 1718.9310655530555,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 26,
'createdAt': datetime.datetime(2021, 7, 1, 8, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 31,
'name': 'Dolly',
'score': 2032.7694382278457,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 26,
'createdAt': datetime.datetime(2021, 7, 1, 9, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 32,
'name': 'Dolly',
'score': 6177.80551327703,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 40,
'createdAt': datetime.datetime(2021, 7, 1, 9, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 33,
'name': 'Dolly',
'score': 6158.845817353103,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 19,
'createdAt': datetime.datetime(2021, 7, 1, 9, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 188624, tzinfo=<UTC>),
'id': 34,
'name': 'Dolly',
'score': 9425.174738105983,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 40,
'createdAt': datetime.datetime(2021, 7, 1, 9, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 35,
'name': 'Dolly',
'score': 2072.13818536118,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 26,
'createdAt': datetime.datetime(2021, 7, 1, 10, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 36,
'name': 'Dolly',
'score': 2551.92916535973,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 42,
'createdAt': datetime.datetime(2021, 7, 1, 10, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 37,
'name': 'Dolly',
'score': 4727.0461537636575,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 40,
'createdAt': datetime.datetime(2021, 7, 1, 10, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 38,
'name': 'Dolly',
'score': 8325.226244350843,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 19,
'createdAt': datetime.datetime(2021, 7, 1, 10, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 39,
'name': 'Dolly',
'score': 9111.140551385512,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 37,
'createdAt': datetime.datetime(2021, 7, 1, 11, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 40,
'name': 'Dolly',
'score': 6631.182480476148,
'status': 1,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 27,
'createdAt': datetime.datetime(2021, 7, 1, 11, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 41,
'name': 'Dolly',
'score': 4179.717633420985,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 41,
'createdAt': datetime.datetime(2021, 7, 1, 11, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 42,
'name': 'Dolly',
'score': 1242.5844800205507,
'status': 2,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 24,
'createdAt': datetime.datetime(2021, 7, 1, 11, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 43,
'name': 'Dolly',
'score': 7948.608503189495,
'status': 5,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 30,
'createdAt': datetime.datetime(2021, 7, 1, 12, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 189471, tzinfo=<UTC>),
'id': 44,
'name': 'Dolly',
'score': 3153.5225301611163,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 26,
'createdAt': datetime.datetime(2021, 7, 1, 12, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 45,
'name': 'Dolly',
'score': 6689.849475511283,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 19,
'createdAt': datetime.datetime(2021, 7, 1, 12, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 46,
'name': 'Dolly',
'score': 7941.698677811398,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 41,
'createdAt': datetime.datetime(2021, 7, 1, 12, 52, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 47,
'name': 'Dolly',
'score': 7124.810788648066,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 19,
'createdAt': datetime.datetime(2021, 7, 1, 13, 7, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 48,
'name': 'Dolly',
'score': 6245.637786368792,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 28,
'createdAt': datetime.datetime(2021, 7, 1, 13, 22, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 49,
'name': 'Dolly',
'score': 1584.2818807777992,
'status': 3,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'},
{'age': 18,
'createdAt': datetime.datetime(2021, 7, 1, 13, 37, 38, 183365, tzinfo=<UTC>),
'generationDate': datetime.datetime(2021, 7, 1, 4, 7, 38, 190470, tzinfo=<UTC>),
'id': 50,
'name': 'Dolly',
'score': 6704.023925214241,
'status': 4,
'testSession': 'f7607f8d-774b-4823-8d47-91c3db056e73'}
]
Notes
this is a very early stage project, just a few functionalities. If you find any bug, please contact me.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for token_parser-0.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a076ed71ce669e81c54ded2b01cfab52e6d8380d125a4a05fa56563243c921b2 |
|
MD5 | 58e4dafeae3549e585d5f0217b44c102 |
|
BLAKE2b-256 | 0b09dd11b2b6b1d6acb18214c557f3d9c08e1754d342f4b2f4528a924deef3b6 |