How to take a screenshot of desktop fast with Java in Windows (ffmpeg, etc.)?

Question

I would like to use java to take a screenshot of my machine using FFMPEG or some other solution. I know linux works with ffmpeg without JNI, but running it in Windows does n

Answer 1

Using the built-in Robots class is way easier than other Java libraries and should probably fit your needs.

If you need a smooth video with >= 30fps (more than 30 screenshots per second), you should first try the Robots approach plus performance improvements there using asynchronous storing of the screenshots.

If it doesn't work for you, try using JNA and that is (even though it's more complex) almost guaranteed to work for smooth screen capturing.

Approach with Robots

The robots class is indeed capable of doing what you want, the problem most screen capturing approaches with Robots have is the saving of the screenshots. An approach could look like that: Looping over the captureScreen() method, grabbing the screen into a BufferedImage, convert it to a byte array and save it with an asynchronous file writer to a target file after adding the future reference of your image to the ArrayList to be able to keep going while storing the image data.

// Pseudo code
while (capturing)
{
    grab bufferedImage (screenCapture) from screen
    convert bufferImage to byte array
    start asynchronous file channel to write to the output file
      and add the future reference (return value) to the ArrayList
}

Approach with JNA

Original Question: How to take screenshots fast in Java?

As it is bad practice to just link, I will post the example here:

import java.awt.Rectangle;
import java.awt.image.BufferedImage;
import java.awt.image.ColorModel;
import java.awt.image.DataBuffer;
import java.awt.image.DataBufferInt;
import java.awt.image.DataBufferUShort;
import java.awt.image.DirectColorModel;
import java.awt.image.Raster;
import java.awt.image.WritableRaster;

import com.sun.jna.Native;
import com.sun.jna.platform.win32.W32API;
import com.sun.jna.win32.W32APIOptions;

public class JNAScreenShot
{

    public static BufferedImage getScreenshot(Rectangle bounds)
    {
        W32API.HDC windowDC = GDI.GetDC(USER.GetDesktopWindow());
        W32API.HBITMAP outputBitmap = GDI.CreateCompatibleBitmap(windowDC, bounds.width, bounds.height);
        try
        {
            W32API.HDC blitDC = GDI.CreateCompatibleDC(windowDC);
            try
            {
                W32API.HANDLE oldBitmap = GDI.SelectObject(blitDC, outputBitmap);
                try
                {
                    GDI.BitBlt(blitDC, 0, 0, bounds.width, bounds.height, windowDC, bounds.x, bounds.y, GDI32.SRCCOPY);
                }
                finally
                {
                    GDI.SelectObject(blitDC, oldBitmap);
                }
                GDI32.BITMAPINFO bi = new GDI32.BITMAPINFO(40);
                bi.bmiHeader.biSize = 40;
                boolean ok = GDI.GetDIBits(blitDC, outputBitmap, 0, bounds.height, (byte[]) null, bi, GDI32.DIB_RGB_COLORS);
                if (ok)
                {
                    GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
                    bih.biHeight = -Math.abs(bih.biHeight);
                    bi.bmiHeader.biCompression = 0;
                    return bufferedImageFromBitmap(blitDC, outputBitmap, bi);
                }
                else
                {
                    return null;
                }
            }
            finally
            {
                GDI.DeleteObject(blitDC);
            }
        }
        finally
        {
            GDI.DeleteObject(outputBitmap);
        }
    }

    private static BufferedImage bufferedImageFromBitmap(GDI32.HDC blitDC, GDI32.HBITMAP outputBitmap, GDI32.BITMAPINFO bi)
    {
        GDI32.BITMAPINFOHEADER bih = bi.bmiHeader;
        int height = Math.abs(bih.biHeight);
        final ColorModel cm;
        final DataBuffer buffer;
        final WritableRaster raster;
        int strideBits = (bih.biWidth * bih.biBitCount);
        int strideBytesAligned = (((strideBits - 1) | 0x1F) + 1) >> 3;
        final int strideElementsAligned;
        switch (bih.biBitCount)
        {
            case 16:
                strideElementsAligned = strideBytesAligned / 2;
                cm = new DirectColorModel(16, 0x7C00, 0x3E0, 0x1F);
                buffer = new DataBufferUShort(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            case 32:
                strideElementsAligned = strideBytesAligned / 4;
                cm = new DirectColorModel(32, 0xFF0000, 0xFF00, 0xFF);
                buffer = new DataBufferInt(strideElementsAligned * height);
                raster = Raster.createPackedRaster(buffer, bih.biWidth, height, strideElementsAligned, ((DirectColorModel) cm).getMasks(), null);
                break;
            default:
                throw new IllegalArgumentException("Unsupported bit count: " + bih.biBitCount);
        }
        final boolean ok;
        switch (buffer.getDataType())
        {
            case DataBuffer.TYPE_INT:
            {
                int[] pixels = ((DataBufferInt) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            case DataBuffer.TYPE_USHORT:
            {
                short[] pixels = ((DataBufferUShort) buffer).getData();
                ok = GDI.GetDIBits(blitDC, outputBitmap, 0, raster.getHeight(), pixels, bi, 0);
            }
                break;
            default:
                throw new AssertionError("Unexpected buffer element type: " + buffer.getDataType());
        }
        if (ok)
        {
            return new BufferedImage(cm, raster, false, null);
        }
        else
        {
            return null;
        }
    }

    private static final User32 USER = User32.INSTANCE;

    private static final GDI32 GDI = GDI32.INSTANCE;

}

interface GDI32 extends com.sun.jna.platform.win32.GDI32
{
    GDI32 INSTANCE = (GDI32) Native.loadLibrary(GDI32.class);

    boolean BitBlt(HDC hdcDest, int nXDest, int nYDest, int nWidth, int nHeight, HDC hdcSrc, int nXSrc, int nYSrc, int dwRop);

    HDC GetDC(HWND hWnd);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, byte[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, short[] pixels, BITMAPINFO bi, int usage);

    boolean GetDIBits(HDC dc, HBITMAP bmp, int startScan, int scanLines, int[] pixels, BITMAPINFO bi, int usage);

    int SRCCOPY = 0xCC0020;
}

interface User32 extends com.sun.jna.platform.win32.User32
{
    User32 INSTANCE = (User32) Native.loadLibrary(User32.class, W32APIOptions.UNICODE_OPTIONS);

    HWND GetDesktopWindow();
}

More information and approaches

• Increasing screen capture speed when using Java and awt.Robot

• http://www.dreamincode.net/forums/topic/234896-faster-screen-capture/

• How to get over 30FPS using Java in a Screen Capture Program?

• http://ffmpeg.org

See also

• http://www.thepcwizard.in/2012/12/java-screen-capturing-tutorial.html

• How to develop screen capture to video application

• http://www.javalobby.org/forums/thread.jspa?threadID=16400&tstart=0

• http://hiddensciencex.blogspot.co.at/2014/01/fast-screen-capture-in-java-example.html

• http://www.coderanch.com/t/340180/GUI/java/efficient-screenshot-Java

• http://www.javaworld.com/article/2071755/learn-java/capture-the-screen.html

• ffmpeg for screen capture?

• Java applet screen capture to a video

• Screen Capture of DirectX programs with Java

Answer 2

Are you familiar with Xuggler? It uses FFmpeg for encoding and decoding. I got to know it a few months ago when I had to extract frames from a video and it worked smoothly.

On the official website you can find some examples including one called "CaptureScreenToFile.java". For more info follow these links:

http://www.xuggle.com/xuggler/

https://github.com/artclarke/xuggle-xuggler/tree/master/src/com/xuggle/xuggler/demos

Answer 3

According to the official ffmpeg documentation you should be able to keep it pretty cross platform if you make the file parameter passed to the FFmpegFrameGrabber (which is really an input parameter that gets passed down as the -i option to ffmpeg) adhere to the different formats each device expects.

ie:

for Windows: dshow expects -i video="screen-capture-recorder"

for OSX: avfoundation expects -i "<screen device index>":

and for Linux: x11grab expects -i :<display id>+<x>,<y>.

So just passing those values (arguments to -i) to the constructor and setting the format (via setFormat) accordingly should do the trick.

Answer 4

You will need to use JNI or JNA to call some combination of CreateCompatibleBitmap, XGetImage, DirectX or OpenGL to grab a screenshot and then copy some raw bitmap data back to Java. My profiling showed a speed up of about 400% over the Robot class when accessing raw bitmap data on X11. I have not tested other platforms at this time. Some very early code is available here but I haven't had much time to work on it recently.