Convert Protobuf-generated Python objects to Pydantic.BaseModel objects with parameter checksum
Project description
protobuf_to_pydantic
Generate the pydantic.Base Model
class (and the corresponding source code) with parameter verification function through the Protobuf file
NOTE:
- Only supports proto3
1.Installation
pip install protobuf_to_pydantic
2.Quick Start
protobuf_to_pydantic
currently has two methods to generate pydantic.BaseModel
objects through Protobuf files,
The first method is to generate the corresponding Python
code file through the Protobuf file in the form of a plugin.
The second method is to generate the corresponding pydantic.BaseModel
object based on the Message
object at runtime.
2.1.Directly generate pydantic.BaseModel
code files through plugins
2.1.1.Use of plugin
The plugin method is the most recommended way to use protobuf-to-pydantic
,
it supports the most complete functions, and it is also very simple to use, assuming that the code corresponding to the Protobuf file is usually generated by the following command:
python -m grpc_tools.protoc -I. example.proto
Then after installing protobuf-to-pydantic
, can use the --protobuf-to-pydantic out
option to use protobuf-to-pydantic
, the command is as follows:
python -m grpc_tools.protoc -I. --protobuf-to-pydantic_out=. example.proto
Note: The
protobuf-to-pydantic
plugin depends onmypy-protobuf
, please installmypy-protobuf
through the commandpython -m pip install protobuf-to-pydanitc[mypy-protobuf]
.
Among them, --protobuf-to-pydantic out=.
indicates the use of the prorobuf-to-pydanitc
plugin, and declares that the output location of the protobuf-to-pydantic
plugin is .
(indicating the use of grpc tools.proto
to use the output path),
In this way, the protobuf-to-pydantic
plugin will write its own generated content in the corresponding file (the file name ends with p2p.py
), such as protobuf-to-pydantic
is example.proto
The generated code file is named example_p2p.py
Note: If
isort
andblack
are installed in the currentPython
environment, thenprotobuf-to-pydantic
will format the generated code throughisort
andblack
.
2.1.2.Plugin configuration
protobuf-to-pydantic
supports configuration functions by reading a Python
file.
Developers first need to create a configuration file in the current path of the running command, the file name is plugin_config.py
, and write the following code:
import logging
from typing import List, Type
from google.protobuf.any_pb2 import Any # type: ignore
from pydantic import confloat, conint
from pydantic.fields import FieldInfo
from protobuf_to_pydantic.gen_model import DescTemplate
# Configure the log output format and log level of the plugin, which is very useful when debugging
logging.basicConfig(format="[%(asctime)s %(levelname)s] %(message)s", datefmt="%y-%m-%d %H:%M:%S", level=logging.DEBUG)
class CustomerField(FieldInfo):
pass
def customer_any() -> Any:
return Any # type: ignore
# For the configuration of the local template, see the use of the local template for details
local_dict = {
"CustomerField": CustomerField,
"confloat": confloat,
"conint": conint,
"customer_any": customer_any,
}
# Specifies the start of key comments
comment_prefix = "p2p"
# Specify the class of the template, you can extend the template by inheriting this class, see the chapter on custom templates for details
desc_template: Type[DescTemplate] = DescTemplate
# Specify the protobuf files of which packages to ignore, and the messages of the ignored packages will not be parsed
ignore_pkg_list: List[str] = ["validate", "p2p_validate"]
Next, change --protobuf-to-pydantic out=.
in the command to --protobuf-to-pydantic out=config path=plugin config.py:.
, as follows:
python -m grpc_tools.protoc -I. --protobuf-to-pydantic_out=config_path=plugin_config.py:. example.proto
Among them, config path=plugin_config.py
on the left side of :
indicates that the configuration file path to be read is plugin_config.py
, and the right side of :
still declares the output of the protobuf-to-pydantic
plugin The position is .
.
In this way, the protobuf-to-pydantic
plugin can be loaded into the configuration file specified by the developer when it is running, and then run according to the configuration defined by the configuration file.
2.2.Generate a pydantic.BaseModel
object at runtime
protobuf_to_pydantic
can generate the corresponding pydantic.BaseModel
object based on the Message
object at runtime。
For example, the UserMessage
in the following Protobuf file named demo.proto
:
// path: ./demo.proto
syntax = "proto3";
package user;
enum SexType {
man = 0;
women = 1;
}
message UserMessage {
string uid=1;
int32 age=2;
float height=3;
SexType sex=4;
bool is_adult=5;
string user_name=6;
}
Through grpc_tools.protoc
, the corresponding Python
code can be generated according to the Protobuf file (the file name at this time is demo_pb2.py
),
and the msg_to_pydantic_model
method of protobuf_to_pydantic
can read the generated Proto file at runtime Message object data,
and generate the corresponding pydantic.BaseModel
object:
from typing import Type
from protobuf_to_pydantic import msg_to_pydantic_model
from pydantic import BaseModel
# import protobuf gen python obj
from . import demo_pb2
UserModel: Type[BaseModel] = msg_to_pydantic_model(demo_pb2.UserMessage)
print(
{
k: v.field_info
for k, v in UserModel.__fields__.items()
}
)
# output
# {
# 'uid': FieldInfo(default='', extra={}),
# 'age': FieldInfo(default=0, extra={}),
# 'height': FieldInfo(default=0.0, extra={}),
# 'sex': FieldInfo(default=0, extra={}),
# 'is_adult': FieldInfo(default=False, extra={}),
# 'user_name': FieldInfo(default='', extra={})
# }
Through the output results, it can be found that the generated pydantic.BaseModel
object also contains uid
, age
, height
, sex
, is adult
and user name
fields, and their corresponding default
The information is consistent with the UserMessage
in the Protobuf file.
In addition to generating the corresponding pydantic.BaseModel
object at runtime, protobuf-to-pydantic
also supports converting the pydantic.BaseModel
object to the corresponding Python
code text at runtime (only compatible with protobuf_to_pydantic
generated pydantic.BaseModel
object).
Among them, the pydantic_model_to_py_code
method of protobuf_to_pydantic
is used to generate code text, and the pydantic_model_to_py_file
method of protobuf_to_pydantic
is used to generate code files,
the sample code of pydantic_model_to_py_file
method of protobuf_to_pydantic
is as follows:
from protobuf_to_pydantic import msg_to_pydantic_model, pydantic_model_to_py_file
# import protobuf gen python obj
from example.example_proto_python_code.example_proto.demo import demo_pb2
pydantic_model_to_py_file(
"./demo_gen_code.py",
msg_to_pydantic_model(demo_pb2.NestedMessage),
)
The code will first convert demo_pb2.NestedMessage
into a pydantic.BaseModel
object, and then the generated object will be converted into the corresponding code content by the pydantic_model_to_py_file
method and written to demo_gen_code.py
file.
It should be noted that if protobuf_to_pydantic
checks that isort
and black
are installed in the current environment, they will be used to format the generated code by default.
2.3.Parameter verification
The Message
object generated according to the Protobuf file will only carry a small amount of information. This is because the ordinary Protobuf file does not have enough parameter verification related information, which requires us to improve the parameter verification information of the Message
object through some additional ways.
Currently protobuf_to_pydantic
supports multiple ways to obtain other information of the Message, so that the generated pydantic.BaseModel
object has the function of parameter verification.
NOTE:
- 1.The text annotation function is not the focus of subsequent function development, and the P2P mode is recommended。
- 2.Plugin mode only supports PGV and P2P mode
2.3.1.Text annotation
Developers can write comments that meet the requirements of protobuf_to_pydantic
for each field in the Protobuf file to provide parameter verification information for protobuf_to_pydantic
, such as the following example:
syntax = "proto3";
package user;
enum SexType {
man = 0;
women = 1;
}
// user info
message UserMessage {
// p2p: {"miss_default": true, "example": "10086"}
// p2p: {"title": "UID"}
string uid=1; // p2p: {"description": "user union id"}
// p2p: {"example": 18, "title": "use age", "ge": 0}
int32 age=2;
// p2p: {"ge": 0, "le": 2.5}
float height=3;
SexType sex=4;
bool is_adult=5;
// p2p: {"description": "user name"}
// p2p: {"default": "", "min_length": 1, "max_length": "10", "example": "so1n"}
string user_name=6;
}
In this example, each annotation that can be used by protobuf_to_pydantic
starts with p2p:
(supports customization) and is followed by a complete Json string. If you are familiar with the usage of pydantic
, you can find This Json string contains the verification information corresponding to pydantic.Field
. For example, the uid
field in UserMessage
contains a total of 4 pieces of information as follows:
Column | Meaning |
---|---|
miss_default | Indicates that the generated field does not have a default value |
example | An example value representing the generated field is 10086 |
title | Indicates that the schema name of the field is UID |
description | The schema documentation for the representation field is described as user_union_id |
Note:
- 1.Currently only single-line comments are supported and comments must be a complete Json data (no line breaks).
- 2.multi line comments are not supported。
When these annotations are written, protobuf_to_pydantic
will bring the corresponding information for each field when converting the Message into the corresponding Pydantic.BaseModel
object, as follows:
from typing import Type
from protobuf_to_pydantic import msg_to_pydantic_model
from pydantic import BaseModel
# import protobuf gen python obj
from example.example_proto_python_code.example_proto.demo import demo_pb2
UserModel: Type[BaseModel] = msg_to_pydantic_model(demo_pb2.UserMessage, parse_msg_desc_method=demo_pb2)
print(
{
k: v.field_info
for k, v in UserModel.__fields__.items()
}
)
# output
# {
# 'uid': FieldInfo(default=PydanticUndefined, title='UID', description='user union id', extra={'example': '10086'}),
# 'age': FieldInfo(default=0, title='use age', ge=0, extra={'example': 18}),
# 'height': FieldInfo(default=0.0, ge=0, le=2, extra={}),
# 'sex': FieldInfo(default=0, extra={}),
# 'is_adult': FieldInfo(default=False, extra={}),
# 'user_name': FieldInfo(default='', description='user name', min_length=1, max_length=10, extra={'example': 'so1n'})
# }
It can be seen from the output results that the output fields carry the corresponding information. In addition, the difference between this code and the above is that the msg_to_pydantic_model
function sets a keyword parameter named parse_msg_desc_method
and its value is demo_pb2
, which enables protobuf_to_pydantic
to obtain additional information for each field in the Message object through comments in the .pyi
file of the demo_pb2
module.
Note:This function requires the use of the mypy-protobuf plugin when generating the corresponding
Python
code from the Protobuf file, and the specified output path of the pyi file is the same as the generatedPython
code path to take effect at the same time.
In addition to obtaining comments through the .pyi
file, protobuf_to_pydantic
also supports setting the value of parse_msg_desc_method
to the root directory path specified when the Message object is generated, so that protobuf_to_pydantic
can parse the comments of the Protobuf file corresponding to the Message object. getting information。
For example, the project structure of the protobuf_to_pydantic
sample code is as follows:
./protobuf_to_pydantic/
├── example/
│ ├── python_example_proto_code/
│ └── example_proto/
├── protobuf_to_pydantic/
└── /
The Protobuf file is stored in the example/example_proto
folder, and then run the following command in the example
directory to generate the Python
code file corresponding to Protobuf:
cd example
python -m grpc_tools.protoc
--python_out=./python_example_proto_code \
--grpc_python_out=./python_example_proto_code \
-I. \
Then the path to be filled in at this time is ./protobuf_to_pydantic/example
, the code is as follows:
from typing import Type
from protobuf_to_pydantic import msg_to_pydantic_model
from pydantic import BaseModel
# import protobuf gen python obj
from example.example_proto_python_code.example_proto.demo import demo_pb2
UserModel: Type[BaseModel] = msg_to_pydantic_model(
demo_pb2.UserMessage, parse_msg_desc_method="./protobuf_to_pydantic/example"
)
print(
{
k: v.field_info
for k, v in UserModel.__fields__.items()
}
)
# output
# {
# 'uid': FieldInfo(default=PydanticUndefined, title='UID', description='user union id', extra={'example': '10086'}),
# 'age': FieldInfo(default=0, title='use age', ge=0, extra={'example': 18}),
# 'height': FieldInfo(default=0.0, ge=0, le=2, extra={}),
# 'sex': FieldInfo(default=0, extra={}),
# 'is_adult': FieldInfo(default=False, extra={}),
# 'user_name': FieldInfo(default='', description='user name', min_length=1, max_length=10, extra={'example': 'so1n'})
# }
From the result, it can be seen that the information carried by the field is the same as the result obtained by the module
NOTE: This method requires lark to be installed in advance and the Protobuf file must exist in the running project.
2.3.2.PGV(protoc-gen-validate)
Currently, the commonly used object validation method in the Protobuf ecosystem is to directly use the protoc-gen-validate project, while protoc-gen-validate project also supports multiple languages, and most Protobuf developers will write pgv
rules once so that different languages support the same validation rules.
And protobuf-to-pydantic
also supports parsing the verification rules of pgv
so that the generated pydantic.BaseModel
class has corresponding verification logic,
It is very simple to use Pgv
verification rules in protobuf_to_pydantic
. First, you need to write the corresponding Pgv
rules in the Protobuf file, and then fill in parse_msg_desc_method
when converting through msg_to_pydantic_model
method The value is PGV
, the code is as follows:
from typing import Type
from protobuf_to_pydantic import msg_to_pydantic_model
from pydantic import BaseModel
# import protobuf gen python obj
from example.example_proto_python_code.example_proto.validate import demo_pb2
UserModel: Type[BaseModel] = msg_to_pydantic_model(
demo_pb2.FloatTest, parse_msg_desc_method="PGV"
)
print(
{
k: v.field_info
for k, v in UserModel.__fields__.items()
}
)
# output
# {
# 'const_test': FieldInfo(default=1.0, const=True, extra={}),
# 'range_e_test': FieldInfo(default=0.0, ge=1, le=10, extra={}),
# 'range_test': FieldInfo(default=0.0, gt=1, lt=10, extra={}),
# 'in_test': FieldInfo(default=0.0, extra={'in': [1.0, 2.0, 3.0]}),
# 'not_in_test': FieldInfo(default=0.0, extra={'not_in': [1.0, 2.0, 3.0]}),
# 'ignore_test': FieldInfo(default=0.0, extra={})
# }
Note:
- 1.For the usage of
Pgv
, see: protoc-gen-validate doc- 2.Need to install
Pgv
throughpip install protoc_gen_validate
Or download validate.proto to the protobuf directory in the project to write pgv rules in the Protobuf file.
2.2.3.P2p
The verification rules of Pgv
are written in the Option attribute of each field of Message
, and there are better code specifications, so the readability of Protobuf files carrying Pgv
verification rules is higher than that of Protobuf carrying comments At the same time, when writing Pgv
rules, you can also experience the convenience brought by IDE auto-completion, but it only supports verification-related logic, and the feature richness is not as good as the file comment mode.
The P2P
mode is an extension of the PGV
mode, which incorporates some functions of text annotations. This mode satisfies the customization of the attributes of each Field
in most pydantic.BaseModel
, such as the following Protobuf file:
syntax = "proto3";
package p2p_validate_test;
import "example_proto/common/p2p_validate.proto";
message FloatTest {
float const_test = 1 [(p2p_validate.rules).float.const = 1];
float range_e_test = 2 [(p2p_validate.rules).float = {ge: 1, le: 10}];
float range_test = 3[(p2p_validate.rules).float = {gt: 1, lt: 10}];
float in_test = 4[(p2p_validate.rules).float = {in: [1,2,3]}];
float not_in_test = 5[(p2p_validate.rules).float = {not_in: [1,2,3]}];
float default_test = 6[(p2p_validate.rules).float.default = 1.0];
float not_enable_test = 7[(p2p_validate.rules).float.enable = false];
float default_factory_test = 8[(p2p_validate.rules).float.default_factory = "p2p@builtin|float"];
float miss_default_test = 9[(p2p_validate.rules).float.miss_default = true];
float alias_test = 10 [(p2p_validate.rules).float.alias = "alias"];
float desc_test = 11 [(p2p_validate.rules).float.description = "test desc"];
float multiple_of_test = 12 [(p2p_validate.rules).float.multiple_of = 3.0];
float example_test = 13 [(p2p_validate.rules).float.example = 1.0];
float example_factory = 14 [(p2p_validate.rules).float.example_factory = "p2p@builtin|float"];
float field_test = 15[(p2p_validate.rules).float.field = "p2p@local|CustomerField"];
float type_test = 16[(p2p_validate.rules).float.type = "p2p@local|confloat"];
float title_test = 17 [(p2p_validate.rules).float.title = "title_test"];
}
protobuf_to_pydantic
can read the generated Message object at runtime and generate a pydantic.BaseModel
object with the corresponding information:
from typing import Type
from protobuf_to_pydantic import msg_to_pydantic_model
from pydantic import BaseModel, confloat
from pydantic.fields import FieldInfo
# import protobuf gen python obj
from example.example_proto_python_code.example_proto.p2p_validate import demo_pb2
class CustomerField(FieldInfo):
pass
DemoModel: Type[BaseModel] = msg_to_pydantic_model(
demo_pb2.FloatTest,
local_dict={"CustomerField": CustomerField, "confloat": confloat},
)
print(
{
k: v.field_info
for k, v in DemoModel.__fields__.items()
}
)
# output:
# {
# 'const_test': FieldInfo(default=1.0, const=True, extra={}),
# 'range_e_test': FieldInfo(default=0.0, ge=1, le=10, extra={}),
# 'range_test': FieldInfo(default=0.0, gt=1, lt=10, extra={}),
# 'in_test': FieldInfo(default=0.0, extra={'in': [1.0, 2.0, 3.0]}),
# 'not_in_test': FieldInfo(default=0.0, extra={'not_in': [1.0, 2.0, 3.0]}),
# 'default_test': FieldInfo(default=1.0, extra={}),
# 'default_factory_test': FieldInfo(default=PydanticUndefined, default_factory=<class 'float'>, extra={}),
# 'miss_default_test': FieldInfo(extra={}),
# 'alias_test': FieldInfo(default=0.0, alias='alias', alias_priority=2, extra={}),
# 'desc_test': FieldInfo(default=0.0, description='test desc', extra={}),
# 'multiple_of_test': FieldInfo(default=0.0, multiple_of=3, extra={}),
# 'example_test': FieldInfo(default=0.0, extra={'example': 1.0}),
# 'example_factory': FieldInfo(default=0.0, extra={'example': <class 'float'>}),
# 'field_test': CustomerField(default=0.0, extra={}),
# 'type_test': FieldInfo(default=0.0, extra={}),
# 'title_test': FieldInfo(default=0.0, title='title_test', extra={})
# }
It is worth noting that this code does not explicitly specify that the value of parse_msg_desc_method
is p2p
, because p2p
is already the default rule of protobuf_to_pydantic
.
Note: See the template chapter for the usage of
local_dict
Note:
- 1.See the template chapter for the usage of
local_dict
- 2.If the reference to the Proto file fails, you need to download p2p_validate.proto in the project and use it in the Protobuf file。
2.3.3.Other parameter support
In addition to the parameters of FieldInfo
, the file comment mode and p2p
mode of protobuf_to_pydantic
also support the following parameters:
- miss_default:By default, the default value of each field in the corresponding
pydantic.BaseModel
object is the same as the default value of each field in the Message, but whenmiss default
istrue
, the setting of the default value will be canceled . - enable: By default,
pydantic.BaseModel
will convert every field in the Message. If some fields do not want to be converted, you can setenable
tofalse
- const: Specifies the value of the field's constant. Note: The const of
pydantic.BaseModel
only supports bool variables. Whenconst
isTrue
, the accepted value can only be the value set bydefault
, and the default value carried by the Message generated by protobuf corresponds to The null value of type does not matchpydantic.BaseModel
, soprotobuf_to_pydantic
makes some changes to the input of this value, but afterconst
sets the value, thecost
property in the generated field isTrue
, and
default` becomes the corresponding value of the setting. - type: To expand the current type, for example, if you want to increase the verification of the bank card number through the
pydantic.types.Payment Card Number
type, you can specify the field type asPayment Card Number
by the following method:message UserPayMessage { string bank_number=1; // p2p: {"type": "p2p@import|pydantic.types|PaymentCardNumber"} }
Note: If you don't know
pydantic
, you can use the following two URLs to learn what parameters Field supports:
2.3.4.Template
In some cases, the value we fill in is a method or function of a certain library in Python
(such as the value of type
parameter and default_factory
parameter), which cannot be realized through Json syntax。
At this time, template parameters can be used to solve the corresponding problems. Currently protobuf_to_pydantic
supports a variety of template parameters。
Note:The
p2p
string at the beginning of the template can be defined by the comment prefix variable
2.3.4.1.p2p@import
This template is used to indicate that the value is a variable under other modules. The specific usage method is as follows:
// comment example
message UserPayMessage {
string bank_number=1; // p2p: {"type": "p2p@import|pydantic.types|PaymentCardNumber"}
}
// p2p example
message UserPayMessage {
string bank_number=1[(p2p_validate.rules).string.type = "p2p@import|pydantic.types|PaymentCardNumber"];
}
// p2p other example
// Since the imported type happens to belong to the `pydantic.types` module, string.pydantic type can be used directly in `p2p` mode
message UserPayMessage {
string bank_number=1[(p2p_validate.rules).string.pydantic_type = "PaymentCardNumber"];
}
The syntax in the format of p2p{template method}|{module to be imported: A}|{variable in the module: B}
is used here, which means that B
object needs to be imported and applied through from A import B
,
Through the definition of the template, protobuf_to_pydantic
will convert the corresponding Message into the following pydantic.BaseModel
:
from pydantic import BaseModel
from pydantic.fields import FieldInfo
# p2p@import|pydantic.types|PaymentCardNumber
from pydantic.types import PaymentCardNumber
class UserPayMessage(BaseModel):
bank_number: PaymentCardNumber = FieldInfo(default="", extra={})
2.3.4.2.p2p@import_instance
The p2p@import
template just imports and uses the variables of a certain library, while p2p@import instance
imports the class of a certain library first,
and finally instantiates it with the specified parameters. The method of use is as follows:
message AnyTest {
google.protobuf.Any default_test = 23 [
(p2p_validate.rules).any.default = 'p2p@import_instance|google.protobuf.any_pb2|Any|{"type_url": "type.googleapis.com/google.protobuf.Duration"}'
];
}
Here is the p2p{template method}|{module to be imported}|{corresponding class}|{corresponding parameter}
syntax, through the definition of the template, protobuf_to_pydantic
will convert the corresponding Message is the following pydantic.BaseModel
object:
from google.protobuf.any_pb2 import Any as AnyMessage
from pydantic import BaseModel
from pydantic.fields import FieldInfo
class AnyTest(BaseModel):
default_test: AnyMessage = FieldInfo(
default=AnyMessage(type_url="type.googleapis.com/google.protobuf.Duration")
)
2.3.4.3.p2p@local
This template is used to introduce user-defined variables. The syntax in the format {template method}|{local variable to be used}
is used here, as follows:
// comment example
message UserPayMessage {
google.protobuf.Timestamp exp=1; // p2p: {"default_factory": "p2p@local|exp_time"}
}
// p2p example
message UserPayMessage {
google.protobuf.Timestamp exp=1[(p2p_validate.rules).timestamp.default_factory= "p2p@local|exp_time"];
}
Then register the corresponding value through the parameter local_dict
when calling the msg_to_pydantic_model
method. The fake code is as follows:
import time
def exp_time() -> float:
return time.time()
msg_to_pydantic_model(
demo_pb2.NestedMessage,
local_dict={"exp_time": exp_time}, # <----
)
In this way, protobuf_to_pydantic
can generate a pydantic.BaseModel
object that meets the requirements:
from datetime import datetime
from pydantic import BaseModel
from pydantic.fields import FieldInfo
from . import exp_time
class UserPayMessage(BaseModel):
exp: datetime = FieldInfo(default_factory=exp_time, extra={})
Note: See the sample code for specific calling and generation methods.
2.3.4.4.p2p@builtin
When the variable to be used comes from a built-in function, this template can be used directly (it can be considered as a simplified version of the p2p@local
template), and the syntax is as follows:
// comment example
message UserPayMessage {
google.protobuf.Timestamp exp=1; // p2p: {"type": "p2p@builtin|float"}
}
// p2p example
message UserPayMessage {
google.protobuf.Timestamp exp=1[(p2p_validate.rules).timestamp.type= "p2p@builtin|float"];
}
In this way, protobuf_to_pydantic
can generate a pydantic.BaseModel
object that meets the requirements:
from pydantic import BaseModel
from pydantic.fields import FieldInfo
class UserPayMessage(BaseModel):
exp: float = FieldInfo()
2.3.4.5.Custom template
Currently, protobuf_to_pydantic
only supports several templates. If you have more template requirements, you can extend the template by inheriting the DescTemplate
class.
For example, there is a strange requirement that the default value of the field is the timestamp when the Message object is generated as a pydantic.BaseModel
object, but the timestamp has two versions, one version has a timestamp of length 10 and the other has a length of 13, so write the following Protobuf file:
message TimestampTest{
int32 timestamp_10 = 1[(p2p_validate.rules).int32.default = "p2p@timestamp|10"];
int32 timestamp_13 = 2[(p2p_validate.rules).int32.default = "p2p@timestamp|13"];
}
This file uses the custom p2p@timestamp|{x}
syntax, where x
only has two values of 10 and 13, and then you can write code according to this template behavior, the code is as follows:
import time
from typing import Any, List
from protobuf_to_pydantic.gen_model import DescTemplate
class CustomDescTemplate(DescTemplate):
def template_timestamp(self, template_var_list: List[str]) -> Any:
timestamp: float = time.time()
length: str = template_var_list[0]
if length == "10":
return int(timestamp)
elif length == "13":
return int(timestamp * 100)
else:
raise KeyError(f"timestamp template not support value:{length}")
from .demo_pb2 import TimestampTest # fake code
from protobuf_to_pydantic import msg_to_pydantic_model
msg_to_pydantic_model(
TimestampTest,
desc_template=CustomDescTemplate
)
This code first creates a CustomDescTemplate
class inherited from DescTemplate
, and this class adds a template_timestamp
method to match the syntax of p2p@timestamp
,
Then specify the template class as CustomDescTemplate
through the desc_template
key parameter in msg_to_pydantic_model
, so that msg_to_pydantic_model
will generate the following code (assuming the code generated when the timestamp is 1600000000 ):
from pydantic import BaseModel
from pydantic.fields import FieldInfo
class TimestampTest(BaseModel):
timestamp_10: int = FieldInfo(default=1600000000)
timestamp_13: int = FieldInfo(default=1600000000000)
3.example
protobuf_to_pydantic
provides some simple sample code, the following is the path of the sample code and protobuf file, just for reference:
Implication | Example Protobuf | Example code |
---|---|---|
Generate Model code with validation rules based on p2p schema | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/example_proto/p2p_validate | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/p2p_validate_example |
Generate the basic Model code | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/example_proto/demo | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/simple_example |
Generate Model code with validation rules from .pyi files | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/example_proto/demo | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/text_comment_example |
Generate Model code with validation rules from protobuf files | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/example_proto/validate | https://github.com/so1n/protobuf_to_pydantic/tree/master/example/validate_example |
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for protobuf_to_pydantic-0.1.6.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | add696717edb83b392d0a476470b9e03ef02daa2ccd356f1e8fd7fcb44693170 |
|
MD5 | 4d48b806c16840f09eb47c474ba2d9f8 |
|
BLAKE2b-256 | fbea238659af0269d660bbe058996f4954b23c2a8e48690ce391b0a4c18cbfc3 |
Hashes for protobuf_to_pydantic-0.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f1f51e458027cfe9e9675580d33035d44487726bbdac7ea2f1ac473a209d7132 |
|
MD5 | 00c099b1a049c207ba33a630575a1232 |
|
BLAKE2b-256 | 1501b1f2e0457ef7ce15493c839100aa746ed8e66b309408678e11543682d5ba |