python receiving image from socket stream

问题

I'm trying to send a frame and throw socket by python, using opencv . The image is sent as string stream but when i remake the image on the server,the bytes are not in the right order.

Client code :

import socket
import numpy as np
import cv2


UDP_IP = "127.0.0.1"
UDP_PORT = 5005

cap = cv2.VideoCapture(1)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
while 1:
    ret, frame = cap.read()
    cv2.imshow('client', frame)

    d = frame.flatten()
    s = d.tostring()
    #sock.sendall(s)
    #print len(s)
    for i in range(20):
        sock.sendto(s[i*46080:(i+1)*46080], (UDP_IP, UDP_PORT))
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

Server side :

import socket
import numpy as np
import cv2


UDP_IP = "127.0.0.1"
UDP_PORT = 5005

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((UDP_IP, UDP_PORT))
s = ""
while True:
    for i in range(20):
        data,addr = sock.recvfrom(46080)
        s = s + data
    #data, addr = sock.recvfrom(46080)
    #s = s+data
    frame = np.fromstring(s, dtype='uint8')
    frame = frame.reshape(480, 640, 3)
    #cv2.imwrite('frame.png',frame)
    cv2.imshow('server', frame)
    s=""
    '''
    if len(data) == 921600:
        frame = np.fromstring(s, dtype='uint8')
        frame = frame.reshape(480, 640, 3)
        cv2.imshow('frame', frame)
        s = ""
    '''
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

frame Client side

frame server side

回答1:

UDP datagrams are not guaranteed to show up in any particular order. On many platforms, localhost UDP will end up ordered anyway, or at least will end up ordered when the machine isn't heavily loaded, but that isn't guaranteed anywhere.

So, if you want to send data over UDP, you need to handle the ordering itself.

This means coming up with a protocol that includes some kind of index or offset within each frame. For example, on the sending side:

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
for framenum in itertools.count():
    # etc.
    #print len(s)
    for i in range(20):
        buf = struct.pack('>II', (framenum, i)) + s[i*46080:(i+1)*46080])
        sock.sendto(buf, (UDP_IP, UDP_PORT))

And on the receiving side:

s = [None]*20
for framenum in itertools.count():
    for i in range(20):
        data,addr = sock.recvfrom(46080)
        frame, index = struct.unpack('>II', data[:8])
        if frame == framenum:
            s[idx] = data[8:]
            if all(s):
                framedata = ''.join(s)
                # now we have a full frame to display

I haven't written code to handle the case where you get data from frame #2 before you've even finished frame #1, because you have to think about what you want to do in that case. Throw out frame 1 and start collecting frame 2? Display what you have of frame 1, filling in the missing gaps with pieces left over from frame 0? Keep a list of frame lists and show each one as soon as it's completed? Keep a list of frame lists and which ones you've already finished so you can make sure to display them in order? Use timestamps so if frame 1 is more than X ms old and you haven't finished it yet, you just throw it out?

If you're doing this over the internet, rather than localhost, there's an additional problem: the last chunk of frame 1 might never arrive. That obviously affects your options—if you're going to wait for all of frame 1 before displaying any later frames, you may just hang forever. You could add a way for the client to ask the server to resend a (frame, i) that it thinks should have already arrived, or a way for the client to acknowledge which ones it has seen so the server can tell that something might have been lost, or maybe a way for the client to tell the server "way to much stuff is coming out of order or getting lost, slow down", …

All of these, and more, are viable designs for a video streaming client. So, you have to pick the one you want, and implement it.

Alternatively, if what you want is to always display the frames in order no matter what, you might want to use TCP instead of UDP. Under the covers, TCP is just sending a bunch of packets with indexes attached, but it has all the logic to put the packets back in order, acknowledge and resend missing packets, slow down when packet loss gets too high (and speed back up again if that was a transient problem), etc., so you don't have to write any of that. What you get is just a stream of bytes, in the exact order they were sent. (Of course that means you have to know how to delimit one message from the next, but in your case, that's trivial: every message is exactly 921600 bytes long, so you'd just keep reading in a loop until you have 921600 bytes.)

来源：https://stackoverflow.com/questions/51921388/python-receiving-image-from-socket-stream

标签

python

image

python-2.7

sockets

OpenCV