How to send getusermedia recorded stream to server nodejs realtime

问题

I am able to send stream blob using socket.io to node js server. But, it is blinking while updating the blob data in video player. I want it to run smooth. How I can send data without blinking of video player. Here is my server code

var express = require("express");
var app = express();
var http = require("http").Server(app);
var io = require("socket.io")(http);
var fs = require("fs")

app.use(express.static('public'))
app.get("/", function(req, res){
 res.sendFile(__dirname+ "/public/index.html");
 });


io.on("connection", function(socket) {
  console.log("A user is connected");
  socket.on("send", function(data){
    console.log(data);
    socket.emit("data", data);
  });
  socket.on("disconnect", function() {
    console.log("A user is disconnected");
  });
});


http.listen(3000, function(){
    console.log("Server is started at port 3000\nTo close use Ctrl+C");
});

And here is my client side code,

<html>
<head><title>Testing</title>
<script src="socket.io/socket.io.js"></script>
<script type="text/javascript" src="MediaStreamRecorder.js"></script> 
</head>
<body>
<video autoplay="true" id="video"></video>
<script type="text/javascript">
var socket = io();
             window.URL.createObjectURL = window.URL.createObjectURL || window.URL.webkitCreateObjectURL || window.URL.mozCreateObjectURL || window.URL.msCreateObjectURL;               
             socket.on("data", function(data){
                    var binaryData = [];
                    binaryData.push(data);
                    videoElement = document.getElementById('video');
                    videoElement.src = window.URL.createObjectURL(new Blob(binaryData, {type: "video/webm"}));
            });

                var mediaConstraints = {
                        video: true
                    };
                navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia || navigator.msGetUserMedia;

                    navigator.getUserMedia(mediaConstraints, onMediaSuccess, onMediaError);

                    function onMediaSuccess(stream) {
                        var arrayOfStreams = [stream];
                        var medias = new MediaStreamRecorder(stream);
                        medias.ondataavailable = function(blob) {
                             socket.emit("send", blob);
                        };
                        medias.start();
                    }

                    function onMediaError(e) {
                        console.error('media error', e);
                    }
</script>
</body>
</html>

I have change timeslice value in MediaStreamRecorder api to 500 default. So, sending data to server after 500 millisecond. But is blinking in the webpage. I have to this to make it real time. Any help will appreciated.

回答1:

But, it is blinking while updating the blob data in video player.

You are changing the .src of the <video> element.

URL.createObjectURL() and MediaStreamRecorder are not necessary.

Instead of changing the .src of <video> element you can pass the MediaStream object once and set <video> element .srcObject to the passed MediaStream.

videoElement = document.getElementById("video");

function onMediaSuccess(stream) {
  if (videoElement.srcObject === null) {
    videoElement.srcObject = stream
  }
}

var videoElement = document.getElementById("video");

videoElement.oncanplay = function() {
  videoElement.play();
}

var media = document.createElement("video");

media.src = "https://nickdesaulniers.github.io/netfix/demo/frag_bunny.mp4";

media.oncanplay = function() {
  media.play();
  var stream = media.captureStream();
  onMediaSuccess(stream);
}

function onMediaSuccess(stream) {
  if (videoElement.srcObject === null) {
    videoElement.srcObject = stream
  }
}

<video id="video"></video>

If requirement is to send a Blob you can use MediaSource, convert Blob to an ArrayBuffer using FileReader or fetch() and append the ArrayBuffer to SourceBuffer, see Unable to stream video over a websocket to Firefox.

回答2:

You're using socket.io which uses TCP connection. If you want to make your application real-time something like skype, you must use UDP connection.

If one frame lags behind then your application should ignore this but in your case you're using TCP connection so it will always deliver in order. Firewall is also an issue for TCP connection. You said that you tried setting up timeout to 500 ms which is too big for real-time application that's why your video is blinking.

If you're willing to give up on TCP connection, I have a small solution for this. I've tried this and it is working fine.

https://www.youtube.com/watch?v=ieBtXwHvoNk

Just one problem is that in this case you cannot send your packets on WAN directly as you can easily do in websockets. You have to implement STUN/TURN or something like that on your server.

If you still have some doubts then see this github issue and read all the replies : https://github.com/socketio/socket.io/issues/1175

I hope this would help.

来源：https://stackoverflow.com/questions/45402263/how-to-send-getusermedia-recorded-stream-to-server-nodejs-realtime

标签

javascript

node.js

Socket.IO

webrtc

mediastream