问题
I'm decoding barcodes using the built in camera, I do this with the capElement.Source.CapturePhotoToStreamAsync to capture photos from the preview. it works, but freezes the app for a brief moment, which feels very clumsy and buggy.
So I'd like to this in background while at least leaving a responsive UI while processing the photos.
So far I came up with this to capture the video stream:
private async void ScanInBackground()
{
bool failedScan = true;
var stream = new InMemoryRandomAccessStream();
await capElement.Source.StartRecordToStreamAsync(MediaEncodingProfile.CreateWmv(VideoEncodingQuality.HD1080p), stream);
while(failedScan)
{
Byte[] bytes = await GetBytesFromStream(stream);
//How to split the bytes into frames?
Task.Delay(50);
}
Dispatcher.RunAsync(CoreDispatcherPriority.Low,() => StopCap());
}
and this method to get the bytes from the stream:
public static async Task<byte[]> GetBytesFromStream(IRandomAccessStream randomStream)
{
var reader = new DataReader(randomStream.GetInputStreamAt(0));
var bytes = new byte[randomStream.Size];
try
{
await reader.LoadAsync((uint)randomStream.Size); reader.ReadBytes(bytes);
}
catch(Exception ex)
{
Logger.LogExceptionAsync(ex, "GetBytesFromStream");
}
return bytes;
}
From the comment at the ScanInBackground, you can see that I have no clue how to split the stream into photos/frames.
回答1:
There is a sample on the Microsoft github page that is relevant, although they target Windows 10. You may be interested in migrating your project to get this functionality.
GetPreviewFrame: This sample will capture preview frames as opposed to full-blown photos. Once it has a preview frame, it can read and edit the pixels on it.
Here is the relevant part:
private async Task GetPreviewFrameAsSoftwareBitmapAsync()
{
// Get information about the preview
var previewProperties = _mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview) as VideoEncodingProperties;
// Create the video frame to request a SoftwareBitmap preview frame
var videoFrame = new VideoFrame(BitmapPixelFormat.Bgra8, (int)previewProperties.Width, (int)previewProperties.Height);
// Capture the preview frame
using (var currentFrame = await _mediaCapture.GetPreviewFrameAsync(videoFrame))
{
// Collect the resulting frame
SoftwareBitmap previewFrame = currentFrame.SoftwareBitmap;
// Add a simple green filter effect to the SoftwareBitmap
EditPixels(previewFrame);
}
}
private unsafe void EditPixels(SoftwareBitmap bitmap)
{
// Effect is hard-coded to operate on BGRA8 format only
if (bitmap.BitmapPixelFormat == BitmapPixelFormat.Bgra8)
{
// In BGRA8 format, each pixel is defined by 4 bytes
const int BYTES_PER_PIXEL = 4;
using (var buffer = bitmap.LockBuffer(BitmapBufferAccessMode.ReadWrite))
using (var reference = buffer.CreateReference())
{
// Get a pointer to the pixel buffer
byte* data;
uint capacity;
((IMemoryBufferByteAccess)reference).GetBuffer(out data, out capacity);
// Get information about the BitmapBuffer
var desc = buffer.GetPlaneDescription(0);
// Iterate over all pixels
for (uint row = 0; row < desc.Height; row++)
{
for (uint col = 0; col < desc.Width; col++)
{
// Index of the current pixel in the buffer (defined by the next 4 bytes, BGRA8)
var currPixel = desc.StartIndex + desc.Stride * row + BYTES_PER_PIXEL * col;
// Read the current pixel information into b,g,r channels (leave out alpha channel)
var b = data[currPixel + 0]; // Blue
var g = data[currPixel + 1]; // Green
var r = data[currPixel + 2]; // Red
// Boost the green channel, leave the other two untouched
data[currPixel + 0] = b;
data[currPixel + 1] = (byte)Math.Min(g + 80, 255);
data[currPixel + 2] = r;
}
}
}
}
}
And declare this outside your class:
[ComImport]
[Guid("5b0d3235-4dba-4d44-865e-8f1d0e4fd04d")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
unsafe interface IMemoryBufferByteAccess
{
void GetBuffer(out byte* buffer, out uint capacity);
}
And of course, your project will have to allow unsafe code for all of this to work.
Have a closer look at the sample to see how to get all the details. Or, to have a walkthrough, you can watch the camera session from the recent //build/ conference, which includes a little bit of a walkthrough through some camera samples.
回答2:
I believe that showing the media preview and handling the different possible exceptions is necessary, here a simple example of how to do that,
let say you have the following UI, with a CaptureElement to show the preview and an Image control to show the captured pic,
mc:Ignorable="d" Loaded="MainPage_OnLoaded">
<Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}" VerticalAlignment="Stretch" HorizontalAlignment="Stretch">
<Grid.ColumnDefinitions>
<ColumnDefinition Width="auto"/>
<ColumnDefinition Width="auto"/>
</Grid.ColumnDefinitions>
<Grid.RowDefinitions>
<RowDefinition Height="auto"/>
<RowDefinition Height="auto"/>
</Grid.RowDefinitions>
<CaptureElement x:Name="PreviewElement" Width="400" Height="400" Grid.Column="0" Grid.Row="0"/>
<Image x:Name="ImageElement" Width="400" Height="400" Grid.Column="1" Grid.Row="0"/>
<Button Click="TakePhoto_Click" Content="Take Photo" Grid.Column="0" Grid.Row="1" Grid.ColumnSpan="2" HorizontalAlignment="Stretch" />
</Grid>
On the code behind declare a mediaCapture field,
private MediaCapture _mediaCapture;
then in the page loaded event handler you need to
- start the media capture device,
- set the several possible exceptions handlers,
and start the cam preview
private async void MainPage_OnLoaded(object sender, RoutedEventArgs e) { //Start the device try { _mediaCapture = new MediaCapture(); _mediaCapture.RecordLimitationExceeded += MediaCapture_RecordLimitationExceeded; _mediaCapture.Failed += MediaCapture_Failed; await _mediaCapture.InitializeAsync(); } catch (UnauthorizedAccessException ex) { (new MessageDialog("Set the permission to use the webcam")).ShowAsync(); } catch (Exception ex) { (new MessageDialog("Can't initialize the webcam !")).ShowAsync(); } //Start the preview if (_mediaCapture != null) { try { PreviewElement.Source = _mediaCapture; await _mediaCapture.StartPreviewAsync(); } catch (Exception ex) { (new MessageDialog("Something went wrong !")).ShowAsync(); } } } private async void MediaCapture_Failed(MediaCapture sender, MediaCaptureFailedEventArgs errorEventArgs) { await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => (new MessageDialog("Media capture failed")).ShowAsync()); } private async void MediaCapture_RecordLimitationExceeded(MediaCapture sender) { await _mediaCapture.StopRecordAsync(); await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, () => (new MessageDialog("Record limitation exceeded")).ShowAsync()); }
And finally here how to properly take a shot, every thing is async so no lag or whatsoever
private async void TakePhoto_Click(object sender, RoutedEventArgs e)
{
if (_mediaCapture != null)
{
try
{
ImageEncodingProperties encodingProperties = ImageEncodingProperties.CreateJpeg();
WriteableBitmap bitmap = new WriteableBitmap((int)ImageElement.Width, (int)ImageElement.Height);
using (var imageStream = new InMemoryRandomAccessStream())
{
await this._mediaCapture.CapturePhotoToStreamAsync(encodingProperties, imageStream);
await imageStream.FlushAsync();
imageStream.Seek(0);
bitmap.SetSource(imageStream);
await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal,
() =>
{
ImageElement.Source = bitmap;
});
}
}
catch (Exception ex)
{
(new MessageDialog("Something went wrong !")).ShowAsync();
}
}
}
来源：https://stackoverflow.com/questions/31578754/winrt-extract-frames-from-video-stream