Near Real Time Video Upload from iPhone

Question

I am trying to find the best way to upload video from an iPhone (iOS5) as fast as possible - real time if possible.

I found this previous question and answer very useful

Answer 1

2) Here's how I chunked the files without dropping too many frames:

- (void) segmentRecording:(NSTimer*)timer {
    AVAssetWriter *tempAssetWriter = self.assetWriter;
    AVAssetWriterInput *tempAudioEncoder = self.audioEncoder;
    AVAssetWriterInput *tempVideoEncoder = self.videoEncoder;
    self.assetWriter = queuedAssetWriter;
    self.audioEncoder = queuedAudioEncoder;
    self.videoEncoder = queuedVideoEncoder;
    //NSLog(@"Switching encoders");

    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
        [tempAudioEncoder markAsFinished];
        [tempVideoEncoder markAsFinished];
        if (tempAssetWriter.status == AVAssetWriterStatusWriting) {
            if(![tempAssetWriter finishWriting]) {
                [self showError:[tempAssetWriter error]];
            }
        }
        if (self.readyToRecordAudio && self.readyToRecordVideo) {
            NSError *error = nil;
            self.queuedAssetWriter = [[AVAssetWriter alloc] initWithURL:[self newMovieURL] fileType:(NSString *)kUTTypeMPEG4 error:&error];
            if (error) {
                [self showError:error];
            }
            self.queuedVideoEncoder = [self setupVideoEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:videoFormatDescription bitsPerSecond:videoBPS];
            self.queuedAudioEncoder = [self setupAudioEncoderWithAssetWriter:self.queuedAssetWriter formatDescription:audioFormatDescription bitsPerSecond:audioBPS];
            //NSLog(@"Encoder switch finished");

        }
    });
}

https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/AVSegmentingAppleEncoder.m

3) Here's a script to concatenate the files on the server

import glob
import os
run = os.system  # convenience alias


files = glob.glob('*.mp4')
out_files = []

n = 0
for file in files:
    out_file = "out-{0}.ts".format(n)
    out_files.append(out_file)
    full_command = "ffmpeg -i {0} -f mpegts -vcodec copy -acodec copy -vbsf h264_mp4toannexb {1}".format(file, out_file)
    run(full_command)
    n += 1

out_file_concat = ''
for out_file in out_files:
    out_file_concat += ' {0} '.format(out_file)

cat_command = 'cat {0} > full.ts'.format(out_file_concat)
print cat_command
run(cat_command)
run("ffmpeg -i full.ts -f mp4 -vcodec copy -acodec copy -absf aac_adtstoasc full.mp4")

https://github.com/chrisballinger/FFmpeg-iOS-Encoder/blob/master/concat-mp4.py

Answer 2

Well you can try to do the buffering on the phone but that seems counter-productive to me, given that it has limited memory. I would try setting up an AVCaptureSession and use the AVCaptureVideoDataOutput which will vend the frames to you on a separate dispatch_queue thread (if setup it will vend them as MPEG frames). That thread can hand the frames off to an async socket to transmit, possibly with a small header that indicates the frame number and video format. Alternately you can hand the data off to a sending thread via a queue which would let you monitor how many frames are waiting to be transmitted.

On the receiving server, you'd want to deal with creating a small buffer (say a few seconds) and doing the frame reordering if they arrive out of order.

The big issue will be detecting the bandwidth and knowing when to drop the quality down so you don't end up with a backlog of packets waiting to go out. That's an entirely different and complicated topic :) The key will be in your selection if codec, quality, and video size... that is going to directly determine the bandwidth required to transmit the frames in real-time. AVVideoCodecH264 is supported in hardware in certain modes and is probably the only realistic option for real-time encoding.

I don't think you are going to find a ready-made example for this though as it represents a lot of work to get working just right.