How do we add selectors/ids to Flutter widgets so they can be accessed from Appium

Question

we want to use Appium/Selenium to do automated testing on a Flutter application. Some elements do not have selectors when viewed in Selenium. In Android we just add ids onto eve

Answer 1

I found an approach with a workaround which then lets you use Selenium reasonably naturally with Flutter Web (although not working with headless browser)

1. You need to find the offset of window x y coordinates from screen x y coordiantes. I found this idea in another thread pageCallibrator.html:

<script>
window.coordinates = [];
document.addEventListener('click', function() {
     window.coordinates = [event.pageX, event.pageY]; 
});
</script>

Then in Selenium setup before running tests (Java example)

    int windowScreenOffsetX = 0;
    int windowScreenOffsetY = 0;

    void callibrateXY(WebDriver driver) {
        driver.get("http://localhost:8080/pageCallibrator.html"); //TODO adjust host
        Dimension size = driver.manage().window().getSize();

        int x = size.width / 2;
        int y = size.height / 2;
        clickMouseAtXY(x, y);
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
        }
        List<Object> coordinates = (List<Object>) ((JavascriptExecutor) driver).executeScript("return window.coordinates;");
        windowScreenOffsetX = x - (int) (long) coordinates.get(0);
        windowScreenOffsetY = y - (int) (long) coordinates.get(1);
    }

Now in Selenium to press a Flutter button

            WebElement continueToBankButtonElement = findElementWithText(driver, "My button text");
            clickMouseAtElement(continueToBankButtonElement);

where you define

import org.openqa.selenium.*

    Robot robot = new Robot();
    Driver driver = new ChromeDriver(options); // TODO handler exceptions and options in a method


    WebElement findElementWithText(WebDriver driver, String text) {
        return driver.findElement(containsTextLocator(text));
    }

    By containsTextLocator(String text) {
        return By.xpath("//*[contains(text(), '" + text + "')]");
    }

    void clickMouseAtElement(WebElement element) {
        clickMouseAtXY(element.getLocation().getX() + element.getSize().width / 2, element.getLocation().getY() + element.getSize().height / 2);
    }

    void clickMouseAtXY(int x, int y) {
        moveMouse(x, y);
        robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
        robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
    }

    /**
     * @param x
     * @param y
     */
    protected void moveMouse(int x, int y) {
        robot.mouseMove(x + windowScreenOffsetX, y + windowScreenOffsetY); // Offset of page from screen
    }

Answer 2

Prior to this morning I knew nothing of Flutter. A few hours later and I can safely say "you don't." While Flutter makes developing an application quick and easy, it removes a lot of the control you have, including the level of customization you're looking for.

There are hits on this on official Flutter message boards dating back a year or two, but there were no answers.

You could attempt locating everything by text? Kluge, difficult or impossible to maintain, but likely your only option at this point.