Render Flutter animation directly to video

Question

Considering that Flutter uses its own graphics engine, is there a way to render Flutter animations directly to video, or create screenshots in a frame by frame fashion?

Answer 1

It's not pretty, but I have managed to get a prototype working. Firstly, all animations need to be powered by a single main animation controller so that we can step through to any part of the animation we want. Secondly, the widget tree that we want to record has to be wrapped inside a RepaintBoundary with a global key. The RepaintBoundary and it's key can produce snapshots of the widget tree as such:

Future _capturePngToUint8List() async {
    // renderBoxKey is the global key of my RepaintBoundary
    RenderRepaintBoundary boundary = renderBoxKey.currentContext.findRenderObject(); 
    // pixelratio allows you to render it at a higher resolution than the actual widget in the application.
    
    ui.Image image = await boundary.toImage(pixelRatio: 2.0);
    ByteData byteData = await image.toByteData(format: ui.ImageByteFormat.png);
    Uint8List pngBytes = byteData.buffer.asUint8List();

    return pngBytes;
  }

The above method can then be used inside a loop which captures the widget tree into pngBytes, and steps the animationController forward by a deltaT specified by the framerate you want as such:

double t = 0;
int i = 1;

setState(() {
  animationController.value = 0.0;
});

Map frames = {};
double dt = (1 / 60) / animationController.duration.inSeconds.toDouble();

while (t <= 1.0) {
  print("Rendering... ${t * 100}%");
  var bytes = await _capturePngToUint8List();
  frames[i] = bytes;

  t += dt;
  setState(() {
    animationController.value = t;
  });
  i++;
}

Finally, all these png frames can be piped into an ffmpeg subprocess to be written into a video. I haven't managed to get this part working nicely yet, so what I've done instead is write out all the png-frames into actual png files and I then manually run ffmpeg inside the folder where they are written. (Note: I've used flutter desktop to be able to access my installation of ffmpeg, but there is a package on pub.dev to get ffmpeg on mobile too)

frames.forEach((key, value) {
  fileWriterFutures.add(_writeFile(bytes: value, location: r"D:\path\to\my\images\folder\" + "frame_$key.png"));
});

await Future.wait(fileWriterFutures);

_runFFmpeg();

Here is my file-writer help-function:

Future _writeFile({@required String location, @required Uint8List bytes}) async {
  File file = File(location);
  return file.writeAsBytes(bytes);
}

And here is my FFmpeg runner function:

void _runFFmpeg() async {
  // ffmpeg -y -r 60 -start_number 1 -i frame_%d.png -c:v libx264 -preset medium -tune animation -pix_fmt yuv420p test.mp4
  var process = await Process.start(
      "ffmpeg",
      [
        "-y", // replace output file if it already exists
        "-r", "60", // framrate
        "-start_number", "1",
        "-i", r"./test/frame_%d.png", // <- Change to location of images
        "-an", // don't expect audio
        "-c:v", "libx264rgb", // H.264 encoding
        "-preset", "medium",
        "-crf",
        "10", // Ranges 0-51 indicates lossless compression to worst compression. Sane options are 0-30
        "-tune", "animation",
        "-preset", "medium",
        "-pix_fmt", "yuv420p",
        r"./test/test.mp4" // <- Change to location of output
      ],
      mode: ProcessStartMode.inheritStdio // This mode causes some issues at times, so just remove it if it doesn't work. I use it mostly to debug the ffmpeg process' output
   );

  print("Done Rendering");
}