How to process VLC UDP stream over OpenCV

Question

I was able to receive/view UDP h264 packets through VLC command line (i.e. VLC --network-caching 0 --demux h264 udp://...)

I am planning on processing those frames t

Answer 1

You can use ffmpeg for streaming.

First test ffmpeg streaming in terminal. In linux, we use v4l2 to grab frames from camera.

Server:

ffmpeg -f v4l2 -i /dev/video0 -preset ultrafast -vcodec libx264 -tune zerolatency -b 900k -f h264 udp://127.0.0.1:5000

Client:

ffplay udp://127.0.0.1:5000

If you're able to view the stream on the client side, then we can use OpenCV for image processing. OpenCV must have ffmepg support. See this link for ffmpeg support check.

    cap = cv2.VideoCapture('udp://127.0.0.1:5000',cv2.CAP_FFMPEG)
    if not cap.isOpened():
        print('VideoCapture not opened')
        exit(-1)

    while True:
        ret, frame = cap.read()

        if not ret:
            print('frame empty')
            break

        cv2.imshow('image', frame)

        if cv2.waitKey(1)&0XFF == ord('q'):
            break

    cap.release()
    cv2.destroyAllWindows()