ubuntu tensorRT5.1.5.0 sample yolov3转onnx转trt，pytorch转onnx转trt推理

tensorRT5.1.5.0安装

官网安装并解压：https://developer.nvidia.com/nvidia-tensorrt-5x-download
添加环境变量：

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/wxy/TensorRT-5.1.5.0/lib

pip安装python版本，装在你的python环境下

cd TensorRT-5.1.5.0/python 
pip install tensorrt-5.1.5.0-cp36-none-linux_x86_64.whl
##验证是否成功
import tensorrt

yolov3转onnx转trt

yolov3转onnx
官方推荐python2，修改一下代码python3也可以运行，yolov3_to_onnx.py修改

1.parse_cfg_file函数里面remainder = cfg_file.read().decode()
2.if sys.version_info[0] > 2:这两句话判断python版本的注释掉
3.download_file函数md5哈希验证取消掉就可以了

执行python yolov3_to_onnx.py

onnx转trt
会出现’NoneType’ object has no attribute ‘serialize’，根据报错排查问题，可能是显存不够，也可能是初始化失败等等。

pytorch转trt推理

总体有两种思路：
1.pytorch转onnx转trt推理，类似于上面yolov3的例子
2.onnx-tensorRT包
pytorch转onnx转trt推理
https://blog.csdn.net/qq_33120609/article/details/96565184?depth_1-utm_source=distribute.pc_relevant.none-task&utm_source=distribute.pc_relevant.none-task
pytorch转onnx

import torch
model = './model_300.pkl'
dummy_input = torch.randn(batch_size, 3, 300, 300, device='cuda')
model = torch.load(model)
torch.onnx.export(model, dummy_input,"mymodel.onnx" , verbose=False)

测试ONNX model 是否与PyTorch model 输出一致

import onnx_tensorrt.backend as backend
import cv2
import onnx
import numpy as np


model = onnx.load("model_300.onnx")
engine = backend.prepare(model, device='CUDA:0')
path = '../Net/test.jpg'
img = cv2.imread(path)
print(img.shape)
img = cv2.resize(img,(300,300))
img = img.transpose(2,0,1)
img = np.ascontiguousarray(img)
img = img[np.newaxis,:]
print(img.shape)
input_data= img.astype(np.float32)
data =np.random.random(size=(3, 300, 300))
#data = data.transpose(2,0,1)
data = data[np.newaxis,:]
#input_data=data.astype(np.float32)
#input_data = np.random.random(size=(1, 3, 300, 300)).astype(np.float32)
output_data = engine.run(input_data)
print(output_data)

onnx-tensorRT包安装
我下载的是tensorRT是5.1.50版本的，注意装v5.1版本的代码，
https://github.com/onnx/onnx-tensorrt
https://github.com/onnx/onnx-tensorrt/tree/5.1

git clone --recursive https://github.com/onnx/onnx-tensorrt.git
git check 55d75dec4f289d1c7a63d5cf578b4c6ab441c03e
git checkout -b branch
mkdir build
cd build
cmake .. -DTENSORRT_ROOT=<tensorrt_install_dir>
OR
cmake .. -DTENSORRT_ROOT=<tensorrt_install_dir> -DGPU_ARCHS="61"
make -j8
sudo make install

onnx2trt model_300.onnx -o my_engine.trt

bug
(1)importError: No module named onnx_tensorrt.backend
去onnx-tensorrt 目录
(2)while trans to ONNX/TensorRT，[8] Assertion failed: get_shape_size(new_shape) == get_shape_size(tensor.getDimensions())
修改变换时view中的 -1
(3)ImportError: /usr/local/lib/libnvonnxparser.so.0: undefined symbol: _ZNK6google8protobuf7Message11GetTypeNameB5cxx11Ev
这可能帮助你
(4) 如果你使用SSD或者类似,maxpool(ceil_mode) 不支持
修改用padding代替

来源：CSDN

作者：三寸光阴___

链接：https://blog.csdn.net/qq_38109843/article/details/104573330

标签

tensorrt

python