When is a WebView ready for a snapshot()?

问题

The JavaFX docs state that a WebView is ready when Worker.State.SUCCEEDED is reached however, unless you wait a while (i.e. Animation, Transition, PauseTransition, etc.), a blank page is rendered.

This suggests that there is an event which occurs inside the WebView readying it for a capture, but what is it?

There's over 7,000 code snippets on GitHub which use SwingFXUtils.fromFXImage but most of them appear to be either unrelated to WebView, are interactive (human masks the race condition) or use arbitrary Transitions (anywhere from 100ms to 2,000ms).

I've tried:

Listening on changed(...) from within the WebView's dimensions (height and width properties DoubleProperty implements ObservableValue, which can monitor these things)
- 🚫Not viable. Sometimes, the value seems to change separate from the paint routine, leading to partial content.
Blindly telling anything and everything to runLater(...) on the FX Application Thread.
- 🚫Many techniques use this, but my own unit tests (as well as some great feedback from other developers) explain that events are often already on the right thread, and this call is redundant. The best I can think of is adds just enough of a delay through queuing that it works for some.
Adding a DOM listener/trigger or JavaScript listener/trigger to the WebView
- 🚫Both JavaScript and the DOM seem to be loaded properly when SUCCEEDED is called despite the blank capture. DOM/JavaScript listeners don't seem to help.
Using an Animation or Transition to effectively "sleep" without blocking the main FX thread.
- ⚠️ This approach works and if the delay is long enough, can yield up to 100% of unit tests, but the Transition times seem to be some future moment that we're just guessing and bad design. For performant or mission-critical applications, this forces the programmer to make a tradeoff between speed or reliability, both a potentially bad experience to the user.

When's a good time to call WebView.snapshot(...)?

Usage:

SnapshotRaceCondition.initialize();
BufferedImage bufferedImage = SnapshotRaceCondition.capture("<html style='background-color: red;'><h1>TEST</h1></html>");
/**
 * Notes:
 * - The color is to observe the otherwise non-obvious cropping that occurs
 *   with some techniques, such as `setPrefWidth`, `autosize`, etc.
 * - Call this function in a loop and then display/write `BufferedImage` to
 *   to see strange behavior on subsequent calls.
 * - Recommended, modify `<h1>TEST</h1` with a counter to see content from
 *   previous captures render much later.
 */

Code Snippet:

import javafx.application.Application;
import javafx.application.Platform;
import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import javafx.concurrent.Worker;
import javafx.embed.swing.SwingFXUtils;
import javafx.scene.Scene;
import javafx.scene.SnapshotParameters;
import javafx.scene.image.WritableImage;
import javafx.scene.web.WebView;
import javafx.stage.Stage;

import java.awt.image.BufferedImage;
import java.io.IOException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.logging.Logger;

public class SnapshotRaceCondition extends Application  {
    private static final Logger log = Logger.getLogger(SnapshotRaceCondition.class.getName());

    // self reference
    private static SnapshotRaceCondition instance = null;

    // concurrent-safe containers for flags/exceptions/image data
    private static AtomicBoolean started  = new AtomicBoolean(false);
    private static AtomicBoolean finished  = new AtomicBoolean(true);
    private static AtomicReference<Throwable> thrown = new AtomicReference<>(null);
    private static AtomicReference<BufferedImage> capture = new AtomicReference<>(null);

    // main javafx objects
    private static WebView webView = null;
    private static Stage stage = null;

    // frequency for checking fx is started
    private static final int STARTUP_TIMEOUT= 10; // seconds
    private static final int STARTUP_SLEEP_INTERVAL = 250; // millis

    // frequency for checking capture has occured 
    private static final int CAPTURE_SLEEP_INTERVAL = 10; // millis

    /** Called by JavaFX thread */
    public SnapshotRaceCondition() {
        instance = this;
    }

    /** Starts JavaFX thread if not already running */
    public static synchronized void initialize() throws IOException {
        if (instance == null) {
            new Thread(() -> Application.launch(SnapshotRaceCondition.class)).start();
        }

        for(int i = 0; i < (STARTUP_TIMEOUT * 1000); i += STARTUP_SLEEP_INTERVAL) {
            if (started.get()) { break; }

            log.fine("Waiting for JavaFX...");
            try { Thread.sleep(STARTUP_SLEEP_INTERVAL); } catch(Exception ignore) {}
        }

        if (!started.get()) {
            throw new IOException("JavaFX did not start");
        }
    }


    @Override
    public void start(Stage primaryStage) {
        started.set(true);
        log.fine("Started JavaFX, creating WebView...");
        stage = primaryStage;
        primaryStage.setScene(new Scene(webView = new WebView()));

        // Add listener for SUCCEEDED
        Worker<Void> worker = webView.getEngine().getLoadWorker();
        worker.stateProperty().addListener(stateListener);

        // Prevents JavaFX from shutting down when hiding window, useful for calling capture(...) in succession
        Platform.setImplicitExit(false);
    }

    /** Listens for a SUCCEEDED state to activate image capture **/
    private static ChangeListener<Worker.State> stateListener = (ov, oldState, newState) -> {
        if (newState == Worker.State.SUCCEEDED) {
            WritableImage snapshot = webView.snapshot(new SnapshotParameters(), null);

            capture.set(SwingFXUtils.fromFXImage(snapshot, null));
            finished.set(true);
            stage.hide();
        }
    };

    /** Listen for failures **/
    private static ChangeListener<Throwable> exceptListener = new ChangeListener<Throwable>() {
        @Override
        public void changed(ObservableValue<? extends Throwable> obs, Throwable oldExc, Throwable newExc) {
            if (newExc != null) { thrown.set(newExc); }
        }
    };

    /** Loads the specified HTML, triggering stateListener above **/
    public static synchronized BufferedImage capture(final String html) throws Throwable {
        capture.set(null);
        thrown.set(null);
        finished.set(false);

        // run these actions on the JavaFX thread
        Platform.runLater(new Thread(() -> {
            try {
                webView.getEngine().loadContent(html, "text/html");
                stage.show(); // JDK-8087569: will not capture without showing stage
                stage.toBack();
            }
            catch(Throwable t) {
                thrown.set(t);
            }
        }));

        // wait for capture to complete by monitoring our own finished flag
        while(!finished.get() && thrown.get() == null) {
            log.fine("Waiting on capture...");
            try {
                Thread.sleep(CAPTURE_SLEEP_INTERVAL);
            }
            catch(InterruptedException e) {
                log.warning(e.getLocalizedMessage());
            }
        }

        if (thrown.get() != null) {
            throw thrown.get();
        }

        return capture.get();
    }
}

Screenshot of the full web page loaded into JavaFX WebView component, not only visible part
Can I capture snapshot of scene programmatically?
Whole page screenshot, Java
JavaFX 2.0+ WebView /WebEngine render web page to an image
Set Height and Width of Stage and Scene in javafx
JavaFX:how to resize the stage when using webview
Correct sizing of Webview embedded in Tabelcell
https://docs.oracle.com/javase/8/javafx/embedded-browser-tutorial/add-browser.htm#CEGDIBBI
http://docs.oracle.com/javafx/2/swing/swing-fx-interoperability.htm#CHDIEEJE
https://bugs.openjdk.java.net/browse/JDK-8126854
https://bugs.openjdk.java.net/browse/JDK-8087569

回答1:

It seems this is a bug which occurs when using WebEngine’s loadContent methods. It also occurs when using load to load a local file, but in that case, calling reload() will compensate for it.

Also, since the Stage needs to be showing when you take a snapshot, you need to call show() before loading the content. Since content is loaded asynchronously, it is entirely possible that it will be loaded before the statement following the call to load or loadContent finishes.

The workaround, then, is to place the content in a file, and call the WebEngine’s reload() method exactly once. The second time the content is loaded, a snapshot can be taken successfully from a listener of the load worker’s state property.

Normally, this would be easy:

Path htmlFile = Files.createTempFile("snapshot-", ".html");
Files.writeString(htmlFile, html);

WebEngine engine = myWebView.getEngine();
engine.getLoadWorker().stateProperty().addListener(
    new ChangeListener<Worker.State>() {
        private boolean reloaded;

        @Override
        public void changed(ObservableValue<? extends Worker.State> obs,
                            Worker.State oldState,
                            Worker.State newState) {
            if (reloaded) {
                Image image = myWebView.snapshot(null, null);
                doStuffWithImage(image);

                try {
                    Files.delete(htmlFile);
                } catch (IOException e) {
                    log.log(Level.WARN, "Couldn't delete " + htmlFile, e);
                }
            } else {
                reloaded = true;
                engine.reload();
            }
        }
    });


engine.load(htmlFile.toUri().toString());

But because you are using static for everything, you’ll have to add some fields:

private static boolean reloaded;
private static volatile Path htmlFile;

And you can use them here:

/** Listens for a SUCCEEDED state to activate image capture **/
private static ChangeListener<Worker.State> stateListener = (ov, oldState, newState) -> {
    if (newState == Worker.State.SUCCEEDED) {
        if (reloaded) {
            WritableImage snapshot = webView.snapshot(new SnapshotParameters(), null);

            capture.set(SwingFXUtils.fromFXImage(snapshot, null));
            finished.set(true);
            stage.hide();

            try {
                Files.delete(htmlFile);
            } catch (IOException e) {
                log.log(Level.WARN, "Couldn't delete " + htmlFile, e);
            }
        } else {
            reloaded = true;
            webView.getEngine().reload();
        }
    }
};

And then you’ll have to reset it each time you load content:

Path htmlFile = Files.createTempFile("snapshot-", ".html");
Files.writeString(htmlFile, html);

Platform.runLater(new Thread(() -> {
    try {
        reloaded = false;
        stage.show(); // JDK-8087569: will not capture without showing stage
        stage.toBack();
        webView.getEngine().load(htmlFile);
    }
    catch(Throwable t) {
        thrown.set(t);
    }
}));

Note that there are better ways to perform multithreaded processing. Instead of using atomic classes, you can simply use volatile fields:

private static volatile boolean started;
private static volatile boolean finished = true;
private static volatile Throwable thrown;
private static volatile BufferedImage capture;

(boolean fields are false by default, and object fields are null by default. Unlike in C programs, this is a hard guarantee made by Java; there is no such thing as uninitialized memory.)

Instead of polling in a loop for changes made in another thread, it’s better to use synchronization, a Lock, or a higher level class like CountDownLatch which uses those things internally:

private static final CountDownLatch initialized = new CountDownLatch(1);
private static volatile CountDownLatch finished;
private static volatile BufferedImage capture;
private static volatile Throwable thrown;
private static boolean reloaded;

private static volatile Path htmlFile;

// main javafx objects
private static WebView webView = null;
private static Stage stage = null;

private static ChangeListener<Worker.State> stateListener = (ov, oldState, newState) -> {
    if (newState == Worker.State.SUCCEEDED) {
        if (reloaded) {
            WritableImage snapshot = webView.snapshot(null, null);
            capture = SwingFXUtils.fromFXImage(snapshot, null);
            finished.countDown();
            stage.hide();

            try {
                Files.delete(htmlFile);
            } catch (IOException e) {
                log.log(Level.WARNING, "Could not delete " + htmlFile, e);
            }
        } else {
            reloaded = true;
            webView.getEngine().reload();
        }
    }
};

@Override
public void start(Stage primaryStage) {
    log.fine("Started JavaFX, creating WebView...");
    stage = primaryStage;
    primaryStage.setScene(new Scene(webView = new WebView()));

    Worker<Void> worker = webView.getEngine().getLoadWorker();
    worker.stateProperty().addListener(stateListener);

    webView.getEngine().setOnError(e -> {
        thrown = e.getException();
    });

    // Prevents JavaFX from shutting down when hiding window, useful for calling capture(...) in succession
    Platform.setImplicitExit(false);

    initialized.countDown();
}

public static BufferedImage capture(String html)
throws InterruptedException,
       IOException {

    htmlFile = Files.createTempFile("snapshot-", ".html");
    Files.writeString(htmlFile, html);

    if (initialized.getCount() > 0) {
        new Thread(() -> Application.launch(SnapshotRaceCondition2.class)).start();
        initialized.await();
    }

    finished = new CountDownLatch(1);
    thrown = null;

    Platform.runLater(() -> {
        reloaded = false;
        stage.show(); // JDK-8087569: will not capture without showing stage
        stage.toBack();
        webView.getEngine().load(htmlFile.toUri().toString());
    });

    finished.await();

    if (thrown != null) {
        throw new IOException(thrown);
    }

    return capture;
}

reloaded is not declared volatile because it is only accessed in the JavaFX application thread.

回答2:

A simpler approach to the Pulse Listener is to use the Callback<SnapshotResult, Void technique (WebView|Node).

public void snapshot(Callback<SnapshotResult, Void>, SnapshotParameters, WritableImage)

Behind the scenes this technique:

Uses an internal TKPulseListener for PostScene
Queues the action if there's more than one pending
Requests a pulse to be fired

... which seems to correct the race condition for the snapshot.

/** Listens for a SUCCEEDED state to activate image capture **/
private static ChangeListener<Worker.State> stateListener = (ov, oldState, newState) -> {
    if (newState == Worker.State.SUCCEEDED) {
        // Register a callback for after the pulse fires
        Callback<SnapshotResult, Void> callback = snapshotResult ->{
            capture.set(SwingFXUtils.fromFXImage(snapshotResult.getImage(), null));
            finished.set(true);
            stage.hide();
            return null;
        };

        webView.snapshot(callback, null, null);
    }
};

Unfortunately, this will only fix the initial blank snapshot problem. Other timing issues still remain, such as resizing the WebView.

回答3:

A proposal I haven't seen before is to listen on JavaFX pulses to determine if WebView has been drawn.

Note, most StackOverflow questions that mention using "pulse" are quickly deterred from using packages in the com.sun namespace so please use with caution...

Initial tests are positive so far. They render the WebView in just around 100ms, varying from around 70-120ms (similar wait times to the quickest working PauseTransition examples). Unfortunately, these positive results may be anecdotal (the wait time may be masking the race condition, not fixing it).

Usage:

/** Listens for a SUCCEEDED state to activate image capture **/
private static ChangeListener<Worker.State> stateListener = (ov, oldState, newState) -> {
    if (newState == Worker.State.SUCCEEDED) {
        try {
            Toolkit.getToolkit().addPostSceneTkPulseListener(new TKPulseListener() {
                @Override
                public void pulse() {
                    WritableImage snapshotResult = webView.snapshot(new SnapshotParameters(), null);
                    capture.set(SwingFXUtils.fromFXImage(snapshotResult, null));
                    finished.set(true);
                    stage.hide();
                    Toolkit.getToolkit().removePostSceneTkPulseListener(this);
                }
            });
// ...

来源：https://stackoverflow.com/questions/59803411/when-is-a-webview-ready-for-a-snapshot

标签

java

multithreading

javafx

race-condition