How do I speed up the YOLO v3-v4 inferencing?

Question

I run YOLO V3 or YOLO v4 on Jeston Xavier, DNN OpenCV version, It is very slow barely reaches ~ 7-9 FPS using DNN in a docker container. I am wondering if you could please tell