UWP - Streaming WebCam over Socket to MediaElement - Broken Picture?

Question

Background

The code I\'ve written records video clips from a webcam, writes them to a memory stream, then transmits the data across a Socke

Answer 1

I'm looking to do something similar (stream video/audio from a UWP app on a Raspberry Pi) but I have been using the simple-communications sample from the Windows 10 SDK which after a bit of tweaking I have been able to get working reliably (there are thread sync issues with the sample code). However the SDK sample uses a proprietary protocol using media extensions and it isn't easy to redirect the stream over the internet which is my use-case so I had a look at your code and got it working (with the same bugs). Simple Real Time communication

A couple of comments on your approach:

1) The RPi can't process video on Win10 very well as it can't use the hardware video encoders so does everything in software. This will cause glitches and I see the CPU performance increasing significantly with over 50% utilisation which means at least one of the CPU cores is working close to max, possibly the one handling the video compression to MP4. However I ran up the SDK sample and got glitch free viewing and about 70% CPU so your problem is likely elsewhere.

2) 5 seconds of latency delay is significant. I get less than 100mSec latency with the real time sample however when I adjusted down your streaming timer to 1 second the breakup was significant and unworkable. Have you thought about changing the design so it streams during capture however I'm not sure the InMemoryRandomAccessStream will let you do that. Another alternative is to capture the preview stream and write a custom media sink to buffer (harder to do as not managed code and likely not able to compress as easily) like the Simple Communication sample does.

3) MP4 is a container not a compression format, and isn't built for streaming as the whole file has to be downloaded before it starts unless the moov metadata record is placed at the beginning of the file. Not sure how UWP handles this, likely your approach of closing off the stream before sending is required to ensure the other end can play it properly.

So not a complete answer but hopefully the above helps.