问题
I have followed this tutorial to retrain MobileNet SSD V1 using Tensorflow GPU as described and got 0.5 loss after training using GPU (below more info about config) and got model.ckpt.
This is the command I used for Training:
python ../models/research/object_detection/legacy/train.py --logtostderr --train_dir=./data/ --pipeline_config_path=./ssd_mobilenet_v1_pets.config
And this is the command for freezing (generate pb file):
python ../models/research/object_detection/export_inference_graph.py --input_type image_tensor --pipeline_config_path ./ssd_mobilenet_v1_pets.config --trained_checkpoint_prefix ./data/model.ckpt-1407 --output_directory ./data/
This is the error I get when I use frozen pb and pbtxt:
Traceback (most recent call last):
File "Object_detection_image.py", line 29, in <module>
cvOut = cvNet.forward()
cv2.error: OpenCV(3.4.3) C:\projects\opencv-python\opencv\modules\dnn\src\dnn.cpp:565: error: (-215:Assertion failed) inputs.size() == requiredOutputs in function 'cv::dnn::experimental_dnn_34_v7::DataLayer::getMemoryShapes'
This is the Object_detection_image.py file I used:
import cv2 as cv
import os
import time
import logging
logger = logging.getLogger()
fh = logging.FileHandler('xyz.log')
fh.setLevel(logging.DEBUG)
logger.addHandler(fh)
cvNet = cv.dnn.readNetFromTensorflow('frozen_inference_graph.pb', 'object_detection.pbtxt')
dir_x = "C:\\Users\\Omen\\Desktop\\LP_dataset\\anno"
for filename in os.listdir(dir_x):
print(filename)
if not (filename.endswith(".png") or filename.endswith(".jpg")):
continue
print('daz')
img = cv.imread(os.path.join(dir_x,filename))
img = cv.resize(img, (300,300))
#cv.imshow('i',img)
#cv.waitKey(0)
img = cv.cvtColor(img,cv.COLOR_BGR2GRAY)
img = cv.cvtColor(img,cv.COLOR_GRAY2RGB)
rows = img.shape[0]
cols = img.shape[1]
#cvNet.setInput(cv.dnn.blobFromImage(img, size=(cols,rows), swapRB=True, crop=False))
cvNet.setInput(cv.dnn.blobFromImage(img, size=(300, 300), crop=False))
t0 = time.time()
cvOut = cvNet.forward()
print(time.time() - t0)
for detection in cvOut[0,0,:,:]:
score = float(detection[2])
#print(score)
if score > 0.80:
left = detection[3] * cols
top = detection[4] * rows
right = detection[5] * cols
bottom = detection[6] * rows
cv.rectangle(img, (int(left), int(top)), (int(right), int(bottom)), (23, 230, 210), thickness=2)
cv.imshow('img', img)
cv.waitKey(0)
This is pbtxt file (also I tried the exported pbtxt and generated pbtxt from pb but not working):
item {
id: 1
name: 'licenseplate'
}
Config:
What is the top-level directory of the model you are using: object_detetion
Have I written custom code: no
OS Platform and Distribution: win10
TensorFlow installed from: binary
TensorFlow GPU version: 1.13.0
CUDA/cuDNN version: 10
GPU model: 1050 GTX
I can provide any files you ask, please help me. In tensorflow's github they told me to ask in Stackoverflow...
Update:
I got the problem solved thanks to the answer, here is the content of cvOut:
[[[[-0.00476191 -0.00361736 0. 0.25361738 -0.07576995
0.03405379 0.40910327]
[ 0.21594621 0.04544836 0. 0.28788495 0.30689242
-0.13025634 0.05074273]
[ 0.46358964 0.19925728 0. -0.09778295 0.26563603
0.34778297 -0.02014329]
[-0.01515752 0.3534766 0. 0.32857144 -0.00361736
0.67142856 0.25361738]
[ 0.25756338 0.03405379 0. 0.21594621 0.3787817
-0.05689242 0.6212183 ]
[ 0.30689242 0.203077 0. 0.796923 0.19925728
0.40103063 -0.09778295]
[ 0.5989694 0.34778297 0. -0.01515752 0.68680996
0.26515752 0.66190475]
[-0.00361736 1.0047619 0. 0.59089667 0.03405379
1.0757699 0.21594621]
[ 0.712115 -0.05689242 0. 0.30689242 0.53641033
0.05074273 1.1302563 ]
[ 0.19925728 0.7343639 0. 0.93230265 0.34778297
0.64652336 -0.01515752]
[ 1.0201433 0.26515752 0. 0.24638264 0.33809522
0.50361735 -0.07576995]
[ 0.2840538 0.40910327 0. 0.04544836 0.19310758
0.28788495 0.5568924 ]
[-0.13025634 0.30074272 0. 0.44925728 0.06769729
0.15221705 0.26563603]
[ 0.59778297 -0.02014329 0. 0.3534766 0.5151575
0.32857144 0.24638264]
[ 0.67142856 0.50361735 0. 0.2840538 0.7424366
0.4659462 0.3787817 ]
[ 0.19310758 0.6212183 0. 0.203077 0.30074272
0.796923 0.44925728]
[ 0.40103063 0.15221705 0. 0.59778297 0.31319004
0.23484248 0.68680996]
[ 0.5151575 0.66190475 0. 1.0047619 0.50361735
0.59089667 0.2840538 ]
[ 1.0757699 0.4659462 0. 0.19310758 0.95455164
0.5568924 0.53641033]
[ 0.30074272 1.1302563 0. 0.7343639 0.15221705
0.93230265 0.59778297]
[ 0.64652336 0.23484248 0. 0.5151575 -0.00476191
0.49638262 0.33809522]
[ 0.75361735 -0.07576995 0. 0.40910327 0.7159462
0.04544836 0.44310758]
[ 0.28788495 0.8068924 0. 0.55074275 0.46358964
0.69925725 0.06769729]
[ 0.40221703 0.26563603 0. -0.02014329 0.48484248
0.3534766 0.7651575 ]
[ 0.32857144 0.49638262 0. 0.75361735 0.25756338
0.5340538 0.7424366 ]
[ 0.7159462 0.3787817 0. 0.6212183 0.8068924
0.203077 0.55074275]
[ 0.796923 0.69925725 0. 0.40221703 0.5989694
0.84778297 0.31319004]
[ 0.48484248 0.68680996 0. 0.66190475 0.49638262
1.0047619 0.75361735]
[ 0.59089667 0.5340538 0. 0.7159462 0.712115
0.44310758 0.95455164]
[ 0.8068924 0.53641033 0. 1.1302563 0.69925725
0.7343639 0.40221703]
[ 0.93230265 0.84778297 0. 0.48484248 1.0201433
0.7651575 -0.00476191]
[ 0.74638265 0.33809522 0. -0.07576995 0.7840538
0.40910327 0.9659462 ]
[ 0.04544836 0.6931076 0. 1.0568924 -0.13025634
0.80074275 0.46358964]
[ 0.94925725 0.06769729 0. 0.26563603 1.0977829
-0.02014329 0.7348425 ]
[ 0.3534766 1.0151576 0. 0.74638265 0.67142856
1.0036174 0.25756338]
[ 0.7840538 0.7424366 0. 0.3787817 0.6931076
0.6212183 1.0568924 ]
[ 0.203077 0.80074275 0. 0.94925725 0.40103063
0.65221703 0.5989694 ]
[ 1.0977829 0.31319004 0. 0.68680996 1.0151576
0.66190475 0.74638265]
[ 1.0047619 1.0036174 0. 0.7840538 1.0757699
0.9659462 0.712115 ]
[ 0.6931076 0.95455164 0. 0.53641033 0.80074275
1.1302563 0.94925725]
[ 0.7343639 0.65221703 0. 1.0977829 0.64652336
0.7348425 1.0201433 ]
[ 1.0151576 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.1 0.2
0.2 0.1 ]
[ 0.1 0.2 0. 0.1 0.1
0.2 0.2 ]
[ 0.1 0.1 0. 0.2 0.1
0.1 0.2 ]
[ 0.2 0.1 0. 0.2 0.2
0.1 0.1 ]
[ 0.2 0.2 0. 0.8479438 0.67317617
0.5581815 0.1778345 ]
[-0.9215721 1.5896183 0. 0.6099795 0.5955366
-0.46569395 -0.8461083 ]
[ 1.6129647 1.4244858 0. 0.5209342 0.17585325
-0.8687666 1.7872683 ]
[ 1.3389692 0.8533131 0. -0.00590521 -0.7195761
1.6236191 1.1828533 ]
[ 1.1838211 0.6728102 0. -0.785988 1.2751837
1.1616383 0.933811 ]
[ 0.4684658 0.2719049 0. 1.2093123 0.66612804
0.66964823 0.55971766]
[ 0.17104894 -1.0688283 0. 0.6494252 0.6844874
0.66586125 0.01329695]
[-1.2607187 -0.22749203 0. -0.8741171 -0.9443728
-0.9659323 -0.03422031]
[-0.0364061 0.54829746 0. 0.6263525 0.66758543
0.04167109 -0.11780822]
[ 0.48400337 0.4685324 0. -0.04594427 0.02469592
-0.3487326 0.08831279]
[ 0.4161314 0.23332608 0. -0.13553022 -0.31008872
0.04969648 0.5674252 ]
[ 0.36492363 -0.07475745 0. -0.03859219 0.2016789
-0.39845943 -0.07058203]
[-0.08173721 0.1720942 0. 0.02323131 0.07122216
0.07469177 0.12792486]
[-0.24689877 0.196296 0. 0.5564647 0.535513
0.22528338 -0.37152448]
[-1.7235181 -1.8204601 0. -1.5040898 -1.8099409
-1.8550183 -1.1855855 ]
[-1.6341007 -1.3448519 0. -1.6656716 -1.6564709
-1.2735447 -1.3357594 ]
[-1.2829769 -1.2869868 0. -1.6657944 -1.4066424
-1.4230443 -1.4196167 ]
[-1.3691044 -1.656098 0. -1.4339573 -1.5685135
-1.633306 -1.4437945 ]]]]
回答1:
The error was caused by the wrong input .pbtxt file passed into the function readNetFromTensorflow because the .pbtxt has to be geneated by tf_text_graph_ssd.py as describe here:
Run this script to get a text graph of SSD model from TensorFlow Object Detection API. Then pass it with .pb file to cv::dnn::readNetFromTensorflow function.
For other models such as faster r-cnn and mask r-cnn, there are also corresponding scripts.
PS: I just found there is a very good official tutorial here.
来源：https://stackoverflow.com/questions/55158339/tensorflow-pb-and-pbtxt-files-dont-work-with-opencv-after-retraining-mobilenet