Multiple HTML5 media elements on one page in iOS (iPad)

Question

My research has led me to learn that Apple\'s media element handler is a singleton, meaning I can\'t have a video playing while an audio is playing in

Answer 1

This is currently not possible. As you notice when a video plays it takes up the full screen with quicktime and moves the browser to the background. The only solution at this time is to merge the audio and video together into an mp4 format and play that single item.

If I understand you correctly, you are not able to merge the audio and video together because it relies on flash? Since iOS can't play flash you should merge the audio and video together and use flash as a backup. There are numerous html5 players which use javascript to try and play the html5 video first then fallback to flash for backup.

Answer 2

How about do it with CSS to do the trick.
Maybe you know about a company called vdopia that distribute video ad on mobile. http://mobile.vdopia.com/index.php?page=mobilewebsolutions
They claim to developed what so called vdo video format, that actually just to do a css sprite running on that :D
I mean you could have your "video" as a framed image, then attach html5 audio tag on there.

I would like to know your response

Answer 3

Could you use a webservice to merge the streams in real time with FFMpeg and then stream one output to quicktime?

To elaborate maybe a library like http://directshownet.sourceforge.net/about.html could also work. It looks like they have method

DESCombine – A class library that uses DirectShow Editing Services to combine video and audio files (or pieces of files) into a single output file. A help file (DESCombine.chm) is provided for using the class.

This could then be used to return the resulting data as the response to the call and loaded via the HTML5 player.

Answer 4

Are you working on a Web App or on a Native Application?

If you are working on a Web App you're in a world of hurt. This is because you simply do not have much control over things that Mobile Safari doesn't provide right away. If this is the case I would come forth and be honest with the stakeholders.

If you are working on a Native Application you can resort to a mechanism that involves some back and forth communication between UIWebView and ObjC. It's actually doable.

The idea is the following:

Insert special <object> elements in your HTML5 documents, that you handcraft yourself according to your needs, taking special care to maintain the attr-* naming convention for non-standard attributes. Here you could insert IDs, paths and other control variables in the multimedia artifacts that you want to play.

Then you could actually build some javascript (on top of jQuery,p.e.) that communicates with ObjC through the delegation mechanism on the UIWebView or through HTTP. I'll go over this choice down below.

Say that on $(document).ready() you go through all the objects that have a special class. A class that you carefully choose to identify all the special <object>. You build a list of such objects and pass them on to the ObjC part of your application. You could easily serialize such list using JSON. Then in ObjC you can do what you want with them. Play them through AVPlayer or some other framework whenever you want them played (again you would resort to a JS - ObjC bridge to actually signal the native part to play a particular element).

You can "communicate" with ObjC through the delegation pattern in UIWebView or through HTTP. You would then have a JS - ObjC bridge in place.

The HTTP approach makes sense in some cases but it involves a lot of extra code and is resource hungry.

If you are building an ObjC application and want further details on how to actually build an ObjC - JS bridge that fits these needs get back to us :)

I'm halting this post as of now because it would be nice to know if it is in fact a Native App.

Cheers.

Answer 5

You mention there is an existing Flash setup of the video - is it a an swf file, could you import it into a video/audio editing software and add an audio track on top?

Something like this: http://www.youtube.com/watch?v=J2vvH7oi8m8&feature=youtube_gdata_player

Also, if it is a Flash file, will you be converting it to an avi or like for iOS? If you'd have to do it anyway, there is your chance for adding an audio track.