Simple Python utility to generate/parse URL query strings, either raw or in Base64 format
Project description
:link: Query String Manager
Simple Python utility to generate/parse URL query strings in standard or in Base64 format
Installation
$ pip install Query-String-Manager
Quick Start
$ python
Python 3.9.0 (default, Oct 27 2020, 14:15:17)
[Clang 12.0.0 (clang-1200.0.32.21)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from query_string_manager import QueryStringManager
# Generate a standard query string from a dictionary
>>> QueryStringManager.generate_query_string({"str": "arg", "int": 1, "float": .01, "bool": True})
'?str=arg&int=1&float=0.01&bool=true'
# Parse a standard query string to a dictionary of Python objects
>>> QueryStringManager.parse_query_string('?str=arg&int=1&float=0.01&bool=true')
{'str': 'arg', 'int': 1, 'float': Decimal('0.01'), 'bool': True}
>>> QueryStringManager.parse('?str=arg&int=1&float=0.01&bool=true')
{'str': 'arg', 'int': 1, 'float': Decimal('0.01'), 'bool': True}
# Generate a base64 encoded query string from a dictionary
>>> QueryStringManager.generate_base64_query_string({"nested_dict": {"float": .1}, "list": [{"int": 1, "bool": True}]})
'?data=eyJuZXN0ZWRfZGljdCI6IHsiZmxvYXQiOiAwLjF9LCAibGlzdCI6IFt7ImludCI6IDEsICJib29sIjogdHJ1ZX1dfQ=='
# Parse a base64 encoded query string to a dictionary
>>> QueryStringManager.parse_base64_query_string('?data=eyJuZXN0ZWRfZGljdCI6IHsiZmxvYXQiOiAwLjF9LCAibGlzdCI6IFt7ImludCI6IDEsICJib29sIjogdHJ1ZX1dfQ==')
{'data': {'nested_dict': {'float': Decimal('0.1')}, 'list': [{'int': 1, 'bool': True}]}}
>>> QueryStringManager.parse('?data=eyJuZXN0ZWRfZGljdCI6IHsiZmxvYXQiOiAwLjF9LCAibGlzdCI6IFt7ImludCI6IDEsICJib29sIjogdHJ1ZX1dfQ==')
{'data': {'nested_dict': {'float': Decimal('0.1')}, 'list': [{'int': 1, 'bool': True}]}}
Overview
This utility can be used easily manage query strings in Python. It allows standard query strings to be generated from Python dictionaries containing the following types (str
, int
, float
decimal.Decimal
, bool
)
>>> QueryStringManager.generate_query_string({"str": "arg", "int": 1, "float": .01, "bool": True})
'?str=arg&int=1&float=0.01&bool=true'
Similarly it can parse query strings into dictionaries of the same types (float
will be replaced with decimal.Decimal
to avoid narrowing/widening issues)
>>> QueryStringManager.parse_query_string('?str=arg&int=1&float=0.01&bool=true')
{'str': 'arg', 'int': 1, 'float': Decimal('0.01'), 'bool': True}
More interestingly, this utility also allows the same functionality but with base64 encoded query strings. This allows more complex objects such as lists and nested dictionaries to be passed in a query string.
For example, a Javascript application could create the following complex query string and base64 encode it:
var obj = {nested: {a: 'a', b: 'b'}, list: [1, {"in": "list"}, true]};
"?data=" + btoa(JSON.stringify(obj));
'?data=eyJuZXN0ZWQiOnsiYSI6ImEiLCJiIjoiYiJ9LCJsaXN0IjpbMSx7ImluIjoibGlzdCJ9LHRydWVdfQ=='
And this library could be used to automatically decode this back to correctly typed objects in Python:
>>> from query_string_manager import QueryStringManager
>>> QueryStringManager.parse_base64_query_string('?data=eyJuZXN0ZWQiOnsiYSI6ImEiLCJiIjoiYiJ9LCJsaXN0IjpbMSx7ImluIjoibGlzdCJ9LHRydWVdfQ==')
{'data': {'nested': {'a': 'a', 'b': 'b'}, 'list': [1, {'in': 'list'}, True]}}
This library can also be used to generate these query strings directly:
>>> from query_string_manager import QueryStringManager
>>> QueryStringManager.generate_base64_query_string({'nested': {'a': 'a', 'b': 'b'}, 'list': [1, {'in': 'list'}, True]}, field_name="data")
'?data=eyJuZXN0ZWQiOiB7ImEiOiAiYSIsICJiIjogImIifSwgImxpc3QiOiBbMSwgeyJpbiI6ICJsaXN0In0sIHRydWVdfQ=='
For the cost of a tiny performance hit, the parse()
method may be used to parse either a base64 encoded or normal format query string, or a query string with a mix of formats:
>>> from query_string_manager import QueryStringManager
>>> QueryStringManager.parse("?val2=false&y=eyJ0ZXN0MiI6IFsxLCAyLCAzXX0=")
{"val2": False, "y": {"test2": [1,2,3]}}
Methods
QueryStringManager.parse()
parse(query_string:str)
Arguments:
-
query_string - The query string to parse into a dictionary. A valid query string will use
"="
to seperate keys and values, like:"?key=value"
. The"?"
prefix is optional in query strings passed to this method.The data in the query string may be in standard or in base64 format. This method will detect the encoding and parse it even if different fields may have different formats
This method can generally be used in place of
parse_base64_query_string()
andparse_query_string()
with the drawback of a slight performance hit checking each the encoding of each field in the query string. See these methods for details on parsing behavior when either format is present
Returns:
- dict - A dict containing the key/value pairs in the query string
Exceptions:
- ValueError - If the passed
query_string
does not have a valid format this exception will be thrown
QueryStringManager.generate_query_string()
generate_query_string(params:dict, safe_chars:str=None)
Arguments:
- params - A dictionary of key/value pairs to write to a query string. This dictionary must be flat and contain no sequences. In addition, the values in the dictionary must be one of the following types: (
str
,int
,float
decimal.Decimal
,bool
). These are the only types that can be cleanly represented in a normal query string
- safe_chars [optional] - When the query string is generated, some characters will be replaced with URL safe characters (such as
" "
to"%20"
). These characters are defined in RFC 3986 and the replacement is performed by urllib.parse.quote. This library specifies some characters to not replace by default (";/?!:@&=+$,."
). Thesafe_chars
argument allows a custom string to be passed defining the characters that should not be replaced by URL safe equivalents
Returns:
- str - The generated query string
Exceptions:
- ValueError - If the constraints on params listed above are not met this exception will be thrown
QueryStringManager.parse_query_string()
parse_query_string(query_string:str, normalize_value:bool=True)
Arguments:
- query_string - The query string to parse into a dictionary. A valid query string will use
"="
to seperate keys and values, like:"?key=value"
. The"?"
prefix is optional in query strings passed to this method. By default, sequencesurllib
detects were replaced for URL safety will be converted to their normal equivalent (such as"%20"
to" "
)
- normalize_value [optional] - By default, data in the query string will be converted to its detected Python type. For example a value of
"1"
in the string will be interpreted as anint
."3.14"
will be interpreted as adecimal.Decimal
andfalse
/true
will be replaced with abool
. Settingnormalize_value
toFalse
will disable this and all values will be interpreted as strings
Returns:
- dict - A dict containing the key/value pairs in the query string
Exceptions:
- ValueError - If the passed
query_string
does not have a valid format this exception will be thrown
QueryStringManager.generate_base64_query_string()
generate_base64_query_string(params:Union[int, str, bool, float, Decimal, list, dict], field_name:str="q")
Arguments:
-
params - An individual value of a serializable type (
str
,int
,float
decimal.Decimal
,bool
), a dictionary (can be nested), list, or any combination of these types valid in Python. This means that non-dict types can be encoded:
>>> QueryStringManager.generate_base64_query_string(3.14) '?q=My4xNA==' >>> QueryStringManager.generate_base64_query_string(True) '?q=dHJ1ZQ=='
It also means lists can be encoded directly, as can nested dicts:
>>> QueryStringManager.generate_base64_query_string([1,2,3]) '?q=WzEsIDIsIDNd' >>> QueryStringManager.generate_base64_query_string({"nested": {"one": {"two": "deep"}}}) '?q=eyJuZXN0ZWQiOiB7Im9uZSI6IHsidHdvIjogImRlZXAifX19' >>> QueryStringManager.generate_base64_query_string([{"dict": 1}]) '?q=W3siZGljdCI6IDF9XQ=='
-
field_name [optional] - The name of the field that should contain the encoded data. The default is
"q"
, creating a query string like"q=<base64 encoded data>"
. If field_name is overridden to something like"data"
the resulting query string would look like"data=<base64 encoded data>"
Returns:
- str - The generated base64 encoded query string
Exceptions:
- ValueError - If the type constraints on params listed above are not met this exception will be thrown
QueryStringManager.parse_base64_query_string()
parse_base64_query_string(query_string:str)
Arguments:
-
query_string - The base64 encoded query string to parse into a dictionary. The passed query string may contain multiple fields and base64 values (seperated by
"&"
), but all must be base64 encoded. The fields will be the top level keys in the dictionary, and the values will be supported Python objects (str
,int
,float
decimal.Decimal
,bool
). The following are examples of decoded query strings:
>>> QueryStringManager.parse_base64_query_string('?q=My4xNA==&test=dHJ1ZQ==&data=WzEsIDIsIDNd') {'q': Decimal('3.14'), 'test': True, 'data': [1, 2, 3]} >>> QueryStringManager.parse_base64_query_string('?q=eyJuZXN0ZWQiOiB7Im9uZSI6IHsidHdvIjogImRlZXAifX19') {'q': {'nested': {'one': {'two': 'deep'}}}} >>> QueryStringManager.parse_base64_query_string('?q=W3siZGljdCI6IDF9XQ==') {'q': [{'dict': 1}]}
Returns:
- dict - A dict containing the key/value pairs in the query string
Exceptions:
- ValueError - If the passed
query_string
does not have a valid format this exception will be thrown
Contributing
- Contributions are welcome! Please not the following when contributing:
-
Unittests must be added under the
tests/
directory for the PR to be approved. You can run unittests from the root project directory with the following command:$ python -m unittest discover -s tests -p test*.py
-
PRs cannot be merged without all unittests passing (they will execute automatically)
-
Merges to
main
will automatically create a new release on PyPi
-
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for Query-String-Manager-1.0.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 383fccdd8c352481b231cf68c7dcecdb4bcdd398aef72dfcd6131b25201baaee |
|
MD5 | 25e8b73f561b8f28aa1ee864d8fe7394 |
|
BLAKE2b-256 | a6ab6a3a18d6e110bcad0961dfdd602319cd94f7f0bb7ac30b9c05c8cd06f639 |
Hashes for Query_String_Manager-1.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7917815b1fcc24fdc74506edd976ee42155f07d01d2497b75713c2e8f4f14a03 |
|
MD5 | d4b3dac26e9a3479160ebd29c76cb890 |
|
BLAKE2b-256 | b502f8af807f08e148a4208166df50d8bb0203a6f2a2cfcf91c435551cf6d83a |