WebView.CapturePreviewToStreamAsync() fails, but only on HoloLens with Unity3D

问题

Currently, we're trying to combine the use of Unity3D holograms and HTML rendering in one app. The idea works as follows:

1) start a XAML app whose main window contains a WebView component (using the XAML app template)
2) Switch to Holographic (DirectX) mode (essentially copy over the code from the Holographic app template)
3) The WebView still runs in the background
4) We can call the WebView.Navigate() method
5) Render the contents of the WebView via the CapturePreviewToStreamAsync() API to a bitmap
6) Convert the bitmap to a texture and display it on the DirectX side- thus having HTML rendering with holograms

Our experiments so far work in:
1) C++/CX
2) C#

However, we have problems if we try to do the same thing with Unity3D. Here is what we do:
1) We create a Unity3D project with mixed reality support
2) We export the Unity project with "UWP Build type" set to "XAML" (instead of D3D). That app builds within MSVS 2017 and starts as a XAML app which automatically switches to holographic space, showing correctly the scenes as defined in Unity.
3) We further edit the main XAML page to contain a WebView element (pointing to a webpage).

Now if we call the CapturePreviewToStreamAsync() API of the WebView component, a System.Exception is thrown with message: Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED))

The error is 100% reproducible and occurs if we use both the Hololens glasses and the emulator.

Any suggestions what might be the cause of the behavior? Seems that Unity creates the main window of the XAML part of the app in a different way, but we don't have access to that since the window creation happens mainly within the re-compiled Unity modules.

Thanks! Martin

回答1:

Although we couldn't find the reason for the exception, we found a workaround, by following the instructions in this blog entry by Jonathan Antoine :

launch a 3D (Holograms) app from a 2D #XAML app and going back to it

The way to make things work was to:
1. Create a new XAML page for the Unity3D generated app from above (let's name it WebViewPage, which contains our WebView component)
2. Modify the Unity generated XAML app to navigate to WebViewPage on startup instead of to the MainPage
3. When needed to switch to holographic space, just navigate to the MainPage the usual way:

    var frame = new Frame();
    Window.Current.Content = frame;
    frame.Navigate(typeof(MainPage));

4. You might also need to move the appCallbacks object from the generated app from the App class to the MainPage class

That solved the problem. Now the CapturePreviewToStreamAsync() call works and returns the data.

NOTE: We noticed that the app leaves the HoloLens emulator (10.0.14393.1358) in some state which doesn't allow the app to be launched a second time without restarting the emulator. But that problem doesn't show up with on a real device

Hope that helps someone
Martin

来源：https://stackoverflow.com/questions/51445940/webview-capturepreviewtostreamasync-fails-but-only-on-hololens-with-unity3d

标签

xaml

unity3d

webview

hololens