This script converts the OpenVINO IR model to Tensorflow's saved_model, tflite, h5 and pb. in (NCHW) format
Project description
openvino2tensorflow
This script converts the OpenVINO IR model to Tensorflow's saved_model, tflite, h5, TensorFlow.js, TF-TRT(TensorRT), CoreML, EdgeTPU and pb.
Work in progress now.
I'm continuing to add more layers of support and bug fixes on a daily basis. If you have a model that you are having trouble converting, please share the .bin
and .xml
with the issue. I will try to convert as much as possible.
1. Environment
- TensorFlow v2.3.1
- OpenVINO 2021.1.110
- Python 3.6+
2. Use case
-
PyTorch (NCHW) -> ONNX (NCHW) -> OpenVINO (NCHW) ->
- ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFLite (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFJS (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TF-TRT (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> CoreML (NHWC)
- ->
-
Caffe (NCHW) -> OpenVINO (NCHW) ->
- ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFLite (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFJS (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TF-TRT (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> CoreML (NHWC)
- ->
-
MXNet (NCHW) -> OpenVINO (NCHW) ->
- ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFLite (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFJS (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TF-TRT (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> CoreML (NHWC)
- ->
-
Keras (NHWC) -> OpenVINO (NCHW・Optimized) ->
- ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFLite (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TFJS (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> TF-TRT (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> EdgeTPU (NHWC) - ->
openvino2tensorflow
-> Tensorflow/Keras (NHWC) -> CoreML (NHWC)
- ->
3. Supported Layers
- Currently, only 4D tensors are supported as input tensors.
- Currently, there are problems with the Reshape operation of 5D Tensor.
No. | OpenVINO Layer | TF Layer | Remarks |
---|---|---|---|
1 | Parameter | Input | Input (4D tensor only) |
2 | Const | Constant, Bias | |
3 | Convolution | Conv2D | |
4 | Add | Add | |
5 | ReLU | ReLU | |
6 | PReLU | PReLU | Maximum(0.0,x)+alpha*Minimum(0.0,x) |
7 | MaxPool | MaxPool2D | |
8 | AvgPool | AveragePooling2D | |
9 | GroupConvolution | DepthwiseConv2D, Conv2D/Split/Concat | |
10 | ConvolutionBackpropData | Conv2DTranspose | |
11 | Concat | Concat | |
12 | Multiply | Multiply | |
13 | Tan | Tan | |
14 | Tanh | Tanh | |
15 | Elu | Elu | |
16 | Sigmoid | Sigmoid | |
17 | HardSigmoid | hard_sigmoid | |
18 | SoftPlus | SoftPlus | |
19 | Swish | Swish | You can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option |
20 | Interpolate | ResizeNearestNeighbor, ResizeBilinear | |
21 | ShapeOf | Shape | |
22 | Convert | Cast | |
23 | StridedSlice | Strided_Slice | |
24 | Pad | Pad, MirrorPad | |
25 | Clamp | ReLU6, Clip | |
26 | TopK | ArgMax, top_k | |
27 | Transpose | Transpose | |
28 | Squeeze | Squeeze | |
29 | Unsqueeze | Identity, expand_dims | WIP |
30 | ReduceMean | reduce_mean | |
31 | ReduceMax | reduce_max | |
32 | ReduceMin | reduce_min | |
33 | ReduceSum | reduce_sum | |
34 | ReduceProd | reduce_prod | |
35 | Subtract | Subtract | |
36 | MatMul | MatMul | |
37 | Reshape | Reshape | |
38 | Range | Range | WIP |
39 | Exp | Exp | |
40 | Abs | Abs | |
41 | SoftMax | SoftMax | |
42 | Negative | Negative | |
43 | Maximum | Maximum | No broadcast |
44 | Minimum | Minimum | No broadcast |
45 | Acos | Acos | |
46 | Acosh | Acosh | |
47 | Asin | Asin | |
48 | Asinh | Asinh | |
49 | Atan | Atan | |
50 | Atanh | Atanh | |
51 | Ceiling | Ceil | |
52 | Cos | Cos | |
53 | Cosh | Cosh | |
54 | Sin | Sin | |
55 | Sinh | Sinh | |
56 | Gather | Gather | |
57 | Divide | Divide, FloorDiv | |
58 | Erf | Erf | |
59 | Floor | Floor | |
60 | FloorMod | FloorMod | |
61 | HSwish | HardSwish | x*ReLU6(x+3)*0.16666667, You can replace swish and hard-swish with each other by using the "--replace_swish_and_hardswish" option |
62 | Log | Log | |
63 | Power | Pow | No broadcast |
64 | Mish | Mish | x*Tanh(softplus(x)) |
65 | Selu | Selu | |
66 | Equal | equal | |
67 | NotEqual | not_equal | |
68 | Greater | greater | |
69 | GreaterEqual | greater_equal | |
70 | Less | less | |
71 | LessEqual | less_equal | |
72 | Select | Select | No broadcast |
73 | LogicalAnd | logical_and | |
74 | LogicalNot | logical_not | |
75 | LogicalOr | logical_or | |
76 | LogicalXor | logical_xor | |
77 | Broadcast | broadcast_to, ones, Multiply | numpy / bidirectional mode, WIP |
78 | Split | Split | |
79 | VariadicSplit | Split, Slice, SplitV | |
80 | MVN | reduce_mean, sqrt, reduce_variance | (x-reduce_mean(x))/sqrt(reduce_variance(x)+eps) |
81 | Result | Identity | Output |
4. Setup
To install using the Python Package Index (PyPI), use the following command.
$ pip3 install openvino2tensorflow --upgrade
To install with the latest source code of the main branch, use the following command.
$ pip3 install git+https://github.com/PINTO0309/openvino2tensorflow --upgrade
5. Usage
usage: openvino2tensorflow [-h] --model_path MODEL_PATH
[--model_output_path MODEL_OUTPUT_PATH]
[--output_saved_model OUTPUT_SAVED_MODEL]
[--output_h5 OUTPUT_H5]
[--output_weight_and_json OUTPUT_WEIGHT_AND_JSON]
[--output_pb OUTPUT_PB]
[--output_no_quant_float32_tflite OUTPUT_NO_QUANT_FLOAT32_TFLITE]
[--output_weight_quant_tflite OUTPUT_WEIGHT_QUANT_TFLITE]
[--output_float16_quant_tflite OUTPUT_FLOAT16_QUANT_TFLITE]
[--output_integer_quant_tflite OUTPUT_INTEGER_QUANT_TFLITE]
[--output_full_integer_quant_tflite OUTPUT_FULL_INTEGER_QUANT_TFLITE]
[--output_integer_quant_type OUTPUT_INTEGER_QUANT_TYPE]
[--string_formulas_for_normalization STRING_FORMULAS_FOR_NORMALIZATION]
[--calib_ds_type CALIB_DS_TYPE]
[--ds_name_for_tfds_for_calibration DS_NAME_FOR_TFDS_FOR_CALIBRATION]
[--split_name_for_tfds_for_calibration SPLIT_NAME_FOR_TFDS_FOR_CALIBRATION]
[--download_dest_folder_path_for_the_calib_tfds DOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDS]
[--tfds_download_flg TFDS_DOWNLOAD_FLG]
[--output_tfjs OUTPUT_TFJS]
[--output_tftrt OUTPUT_TFTRT]
[--output_coreml OUTPUT_COREML]
[--output_edgetpu OUTPUT_EDGETPU]
[--replace_swish_and_hardswish REPLACE_SWISH_AND_HARDSWISH]
[--optimizing_hardswish_for_edgetpu OPTIMIZING_HARDSWISH_FOR_EDGETPU]
[--replace_prelu_and_minmax REPLACE_PRELU_AND_MINMAX]
[--yolact] [--debug]
[--debug_layer_number DEBUG_LAYER_NUMBER]
optional arguments:
-h, --help show this help message and exit
--model_path MODEL_PATH
input IR model path (.xml)
--model_output_path MODEL_OUTPUT_PATH
The output folder path of the converted model file
--output_saved_model OUTPUT_SAVED_MODEL
saved_model output switch
--output_h5 OUTPUT_H5
.h5 output switch
--output_weight_and_json OUTPUT_WEIGHT_AND_JSON
weight of h5 and json output switch
--output_pb OUTPUT_PB
.pb output switch
--output_no_quant_float32_tflite OUTPUT_NO_QUANT_FLOAT32_TFLITE
float32 tflite output switch
--output_weight_quant_tflite OUTPUT_WEIGHT_QUANT_TFLITE
weight quant tflite output switch
--output_float16_quant_tflite OUTPUT_FLOAT16_QUANT_TFLITE
float16 quant tflite output switch
--output_integer_quant_tflite OUTPUT_INTEGER_QUANT_TFLITE
integer quant tflite output switch
--output_full_integer_quant_tflite OUTPUT_FULL_INTEGER_QUANT_TFLITE
full integer quant tflite output switch
--output_integer_quant_type OUTPUT_INTEGER_QUANT_TYPE
Input and output types when doing Integer Quantization
('int8 (default)' or 'uint8')
--string_formulas_for_normalization STRING_FORMULAS_FOR_NORMALIZATION
String formulas for normalization. It is evaluated by
Pythons eval() function. Default: '(data - [127.5,127.5,127.5]) / [127.5,127.5,127.5]'
--calib_ds_type CALIB_DS_TYPE
Types of data sets for calibration. tfds or
numpy(Future Implementation)
--ds_name_for_tfds_for_calibration DS_NAME_FOR_TFDS_FOR_CALIBRATION
Dataset name for TensorFlow Datasets for calibration.
https://www.tensorflow.org/datasets/catalog/overview
--split_name_for_tfds_for_calibration SPLIT_NAME_FOR_TFDS_FOR_CALIBRATION
Split name for TensorFlow Datasets for calibration.
https://www.tensorflow.org/datasets/catalog/overview
--download_dest_folder_path_for_the_calib_tfds DOWNLOAD_DEST_FOLDER_PATH_FOR_THE_CALIB_TFDS
Download destination folder path for the calibration
dataset. Default: $HOME/TFDS
--tfds_download_flg TFDS_DOWNLOAD_FLG
True to automatically download datasets from
TensorFlow Datasets. True or False
--output_tfjs OUTPUT_TFJS
tfjs model output switch
--output_tftrt OUTPUT_TFTRT
tftrt model output switch
--output_coreml OUTPUT_COREML
coreml model output switch
--output_edgetpu OUTPUT_EDGETPU
edgetpu model output switch
--replace_swish_and_hardswish REPLACE_SWISH_AND_HARDSWISH
Replace swish and hard-swish with each other
--optimizing_hardswish_for_edgetpu OPTIMIZING_HARDSWISH_FOR_EDGETPU
Optimizing hardswish for edgetpu
--replace_prelu_and_minmax REPLACE_PRELU_AND_MINMAX
Replace prelu and minimum/maximum with each other
--yolact Specify when converting the Yolact model
--debug debug mode switch
--debug_layer_number DEBUG_LAYER_NUMBER
The last layer number to output when debugging. Used
only when --debug=True
usage: pb_to_saved_model [-h] --pb_file_path PB_FILE_PATH
--inputs INPUTS
--outputs OUTPUTS
[--model_output_path MODEL_OUTPUT_PATH]
optional arguments:
-h, --help show this help message and exit
--pb_file_path PB_FILE_PATH
Input .pb file path (.pb)
--inputs INPUTS (e.g.1) input:0,input:1,input:2 / (e.g.2)
images:0,input:0,param:0
--outputs OUTPUTS (e.g.1) output:0,output:1,output:2 / (e.g.2)
Identity:0,Identity:1,output:0
--model_output_path MODEL_OUTPUT_PATH
The output folder path of the converted model file
usage: pb_to_tflite.py [-h] --pb_file_path PB_FILE_PATH --inputs INPUTS
--outputs OUTPUTS
[--model_output_path MODEL_OUTPUT_PATH]
optional arguments:
-h, --help show this help message and exit
--pb_file_path PB_FILE_PATH
Input .pb file path (.pb)
--inputs INPUTS (e.g.1) input,input_1,input_2 / (e.g.2)
images,input,param
--outputs OUTPUTS (e.g.1) output,output_1,output_2 / (e.g.2)
Identity,Identity_1,output
--model_output_path MODEL_OUTPUT_PATH
The output folder path of the converted model file
usage: ir_weight_extractor.py [-h] -m MODEL -o OUTPUT_PATH
optional arguments:
-h, --help show this help message and exit
-m MODEL, --model MODEL
input IR model path
-o OUTPUT_PATH, --output_path OUTPUT_PATH
weights output folder path
6. Execution sample
6-1. Conversion of OpenVINO IR to Tensorflow models
OutOfMemory may occur when converting to saved_model or h5 when the file size of the original model is large, please try the conversion to a pb file alone.
$ openvino2tensorflow \
--model_path=openvino/448x448/FP32/Resnet34_3inputs_448x448_20200609.xml \
--output_saved_model True \
--output_pb True \
--output_weight_quant_tflite True \
--output_float16_quant_tflite True \
--output_no_quant_float32_tflite True
6-2. Convert Protocol Buffer (.pb) to saved_model
This tool is useful if you want to check the internal structure of pb files, tflite files, .h5 files, coreml files and IR (.xml) files. https://lutzroeder.github.io/netron/
$ pb_to_saved_model \
--pb_file_path model_float32.pb \
--inputs inputs:0 \
--outputs Identity:0
6-3. Convert Protocol Buffer (.pb) to tflite
$ pb_to_tflite \
--pb_file_path model_float32.pb \
--inputs inputs \
--outputs Identity,Identity_1,Identity_2
6-4. Converts saved_model to OpenVINO IR
$ python3 ${INTEL_OPENVINO_DIR}/deployment_tools/model_optimizer/mo_tf.py \
--saved_model_dir saved_model \
--output_dir openvino/reverse
6-5. Checking the structure of saved_model
$ saved_model_cli show \
--dir saved_model \
--tag_set serve \
--signature_def serving_default
7. Output sample
8. Model Structure
https://digital-standard.com/threedpose/models/Resnet34_3inputs_448x448_20200609.onnx
ONNX | OpenVINO | TFLite |
---|---|---|
9. My article
10. Conversion Confirmed Models
- u-2-net
- mobilenet-v2-pytorch
- midasnet
- footprints
- efficientnet-b0-pytorch
- efficientdet-d0
- dense_depth
- deeplabv3
- colorization-v2-norebal
- age-gender-recognition-retail-0013
- resnet
- arcface
- emotion-ferplus
- mosaic
- retinanet
- shufflenet-v2
- squeezenet
- version-RFB-320
- yolov4
- yolov4x-mish
- ThreeDPoseUnityBarracuda - Resnet34_3inputs_448x448
- efficientnet-lite4
- nanodet
- yolov4-tiny
- yolov5s
- yolact
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for openvino2tensorflow-1.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 051732d5a72f50dbd10593fed17fcfbf04765b9a891709d8f9e53394ae37c99b |
|
MD5 | dd39840ffb4fe3426023b6d707e076b1 |
|
BLAKE2b-256 | c3a2e5285a69f369ff70b9294ccf6c78c85c0b6ab28034aac54f41b8f1450371 |
Hashes for openvino2tensorflow-1.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f71f5061298c43163f3eeb44b2222bae95635a0e138df5a02c6f98946b4e26fd |
|
MD5 | 1b0e724008489e05c72b6def32f07210 |
|
BLAKE2b-256 | d2404dd73e0545d9937682c198364a81c01a5796b816a6c39675003b021a0791 |