Skip frames and seek to end of RTSP stream in OpenCV

问题

I capture and process an IP camera RTSP stream in a OpenCV 3.4.2 on Raspberry Pi. Unfortunately the processing takes quite a lot of time, roughly 0.2s per frame, and the stream quickly gets delayed.

I don't mind if I skip some frames so I'm looking for a way to seek to the end of the stream before capturing and processing the next frame.

vcap = cv2.VideoCapture("rtsp://{IPcam}/12")

while(1):
    ret, frame = vcap.read()
    time.sleep(0.2)              # <= Simulate processing time
    cv2.imshow('VIDEO', frame)
    if cv2.waitKey(1) == 27:
        break
    vcap.seek_to_end()           # <== How to do this?

How can I do that vcap.seek_to_end() to catch up with the stream, discard the missed frames, and start processing the most current one?

Thanks!

回答1:

Try this:

vcap = cv2.VideoCapture("rtspsrc location=rtsp://{IPcam}/12 ! decodebin ! videoconvert ! appsink max-buffers=1 drop=true")

This uses gstreamer to grab your camera feed, and will maintain a buffer of length 1 and drop the oldest as new incoming frames are received. Then, every time you call vcap.read() you should get the latest frame.

You can also try using the OMX decoder on the Raspberry Pi if you notice CPU usage is really high, as this will decode the video (assuming it's h264) on the GPU: ! rtph264depay ! h264parse ! omxh264dec ! appsink max-buffers=1 drop=true

You may need to recompile OpenCV as by default it's compiled with FFMPEG support, not gstreamer. This is fairly simple, just pass -D WITH_GSTREAMER=ON -D WITH_FFMPEG=OFF to the cmake command. Make sure you have the gstreamer development libs installed apt install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev.

回答2:

I rewrote threading implementation from Shubham into a class, to be able to stream many cameras:

import threading
from threading import Lock
import cv2

class Camera:
    last_frame = None
    last_ready = None
    lock = Lock()

    def __init__(self, rtsp_link):
        capture = cv2.VideoCapture(rtsp_link)
        thread = threading.Thread(target=self.rtsp_cam_buffer, args=(capture,), name="rtsp_read_thread")
        thread.daemon = True
        thread.start()

    def rtsp_cam_buffer(self, capture):
        while True:
            with self.lock:
                self.last_ready, self.last_frame = capture.read()


    def getFrame(self):
        if (self.last_ready is not None) and (self.last_frame is not None):
            return self.last_frame.copy()
        else:
            return None

Then, it can be used as:

capture = Camera('rtsp://...')

while True:
    frame = capture.getFrame()

回答3:

I managed a hack around it by creating a read thread which puts the frame in a variable and the application uses

import threading 
from threading import Lock
import cv2


rtsp_link = "rtsp://url"
vcap = cv2.VideoCapture(rtsp_link)

latest_frame = None
last_ret = None
lo = Lock()

def rtsp_cam_buffer(vcap):
    global latest_frame, lo, last_ret
    while True:
        with lo:
            last_ret, latest_frame = vcap.read()


t1 = threading.Thread(target=rtsp_cam_buffer,args=(vcap,),name="rtsp_read_thread")
t1.daemon=True
t1.start()


while True :
    if (last_ret is not None) and (latest_frame is not None):
        img = latest_frame.copy()
    else:
        print("unable to read the frame")
        time.sleep(0.2)
        continue

Its not the best way to do, but it solves the purpose.

回答4:

Adding max-buffers=1 drop=true to the pipeline like so pipeline="rtspsrc location=rtsp://camera_ip_address latency=10 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink max-buffers=1 drop=true" works for me

来源：https://stackoverflow.com/questions/51722319/skip-frames-and-seek-to-end-of-rtsp-stream-in-opencv

标签

python

OpenCV

video-capture

rtsp