不那么乐的世界

CenterNet调试记录

 Tech  1151  3 分钟

CenterNet工程调试的过程及问题

最近调试了关于CenterNet的几个工程

  • CenterNet
  • RTM3D
  • tensorrt_demos
  • TensorRT-CenterNetm
  • TensorRT-3D
  • TensorRT-Yolov3

1. CenterNet

CenterNet Gitlink

作者提供了基于三种特征提取模型的训练集:

  • Hourglass104
  • DLA-34
  • ResNet101

Q1 ModuleNotFoundError: No module named '_ext' 主要的问题是在DCNv2需要重新编译 其目录在/src/lib/models/networks/DCNv2

但是安装其中make.sh并未编译成功,而后发现是pytorch版本问题 ,随后更新pytorch版本至1.4.0 再次make.sh成功

随后在Model_zoo下载对应的model进行调试

Model Zoo

CenterNet提供了三个数据集:

  • ctdet_coco_dla_2x 用于detection
  • multi_pose_dla_3x 用于human pose estimation
  • ddd_3dop 用于3D目标检测

用于检测: python demo.py ctdet --demo /path/to/image/or/folder/or/video --load_model ../models/ctdet_coco_dla_2x.pth

同时提供了视频检测接口: python demo.py ctdet --demo webcam --load_model ../models/ctdet_coco_dla_2x.pth

2.RTM3D

RTM3D在编译时,遇到问题 Xavier上缺少MAGMA library 目前还没有找到在Xavier上的编译方法

RTM3D GitLink

3.tensorrt_demos

tensorrt_demos GitLink

GoogleNet与SSD编译 运行都没有问题

Yolov3 Yolov4 由于TensorRT版本不一致 还没有尝试

4.TensorRT-CenterNet

tensorRT_centerNet Gitlink 在Xavier上运行

pytorch 1.4.0
ubuntu 1804
TensorRT 5.1
onnx-tensorrt v4
cuda 10.0

ctder2onnx

step 1

利用THC-basedDCNv2替代ATen-basedDCNv2. https://github.com/CaoWGG/TensorRT-CenterNet/tree/master/readme/dcn 在CenterNet工程中copy dcn cp -r dcn lib/models/networks 编译dcn python setup.py build_ext --inplace

step2

添加symbolic to DeformConvFunction 地址在readme/dcn/functions/deform_conv.py中,但是经过比对,发现拷贝的代码已经添加了

step3

修改import的文件

change (from .DCNv2.dcn_v2 import DCN) to (from .dcn.modules.deform_conv import ModulatedDeformConvPack as DCN) in pose_dla_dcn.py and resnet_dcn.py

step4

进行转换

利用Trans.py 进行转换,转换代码如下:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

from lib.opts import opts
from lib.models.model import create_model, load_model
from types import MethodType
import torch.onnx as onnx
import torch
from torch.onnx import OperatorExportTypes
from collections import OrderedDict
## onnx is not support dict return value
## for dla34
def pose_dla_forward(self, x):
    x = self.base(x)
    x = self.dla_up(x)
    y = []
    for i in range(self.last_level - self.first_level):
        y.append(x[i].clone())
    self.ida_up(y, 0, len(y))
    ret = []  ## change dict to list
    for head in self.heads:
        ret.append(self.__getattr__(head)(y[-1]))
    return ret
## for dla34v0
def dlav0_forward(self, x):
    x = self.base(x)
    x = self.dla_up(x[self.first_level:])
    # x = self.fc(x)
    # y = self.softmax(self.up(x))
    ret = []  ## change dict to list
    for head in self.heads:
        ret.append(self.__getattr__(head)(x))
    return ret
## for resdcn
def resnet_dcn_forward(self, x):
    x = self.conv1(x)
    x = self.bn1(x)
    x = self.relu(x)
    x = self.maxpool(x)

    x = self.layer1(x)
    x = self.layer2(x)
    x = self.layer3(x)
    x = self.layer4(x)
    x = self.deconv_layers(x)
    ret = []  ## change dict to list
    for head in self.heads:
        ret.append(self.__getattr__(head)(x))
    return ret

forward = {'dla':pose_dla_forward,'dlav0':dlav0_forward,'resdcn':resnet_dcn_forward}

opt = opts().init()  ## change lib/opts.py add_argument('task', default='ctdet'....) to add_argument('--task', default='ctdet'....)
opt.arch = 'dla_34'
opt.heads = OrderedDict([('hm', 80), ('reg', 2), ('wh', 2)])
opt.head_conv = 256 if 'dla' in opt.arch else 64
print(opt)
model = create_model(opt.arch, opt.heads, opt.head_conv)
model.forward = MethodType(forward[opt.arch.split('_')[0]], model)
load_model(model, 'ctdet_coco_dla_2x.pth')
model.eval()
model.cuda()
input = torch.zeros([1, 3, 512, 512]).cuda()
onnx.export(model, input, "ctdet_coco_dla_2x.onnx", verbose=True,
            operator_export_type=OperatorExportTypes.ONNX)

通过代码将.pth文件转换为.onnx文件.

对TensorRT-CenterNet进行编译

git clone https://github.com/CaoWGG/TensorRT-CenterNet.git
cd TensorRT-CenterNet
mkdir build
cd build && cmake .. && make
cd ..

通过buildEngine建立onnx到engine的转换

./buildEngine -i model/ctdet_coco_dla_2x.onnx -o model/ctdet_coco_dla_2x.engine 
./runDet -e model/ctdet_coco_dla_2x.engine -i test.jpg -c test.h264

5.TensorRT-CenterNet-3D

TensorRT-CenterNet-3D Gitlink

下载ddd_3dop.pth,利用CenterNet-3D转换为onnx,参考ctder2onnx的步骤 利用netron工具检查输出onnx model is (('hm', 3), ('dep', 1), ('rot', 8), ('dim', 3), ('wh', 2), ('reg', 2)) 完成编译

cd TensorRT-CenterNet-3D
mkdir build
cd build && cmake .. && make
cd ..

生成engine文件,完成推理

./buildEngine -i model/ddd_3dop.onnx -o model/ddd_3dop.engine
./runDet -e model/ddd_3dop.engine -i 000292.png -c test.h264

With the help of TensorRT, the inference time has been shortened from 0.227s to 98.89ms.

updatedupdated2021-05-272021-05-27

相关文章：