Unable to tap (x,y) coordinate in landscape mode

Question

In Xcode 8 / Swift 3, using the coordinate(withNormalizedOffset: CGVector) function to interact with an XCUIElement appears to work only in portrait mode.

To test this functionality, I created a single screen project with a button centered in the view. I then ran the following UI test:

func testExample() {

    XCUIDevice.shared().orientation = .portrait

    let window = XCUIApplication().windows.element(boundBy: 0)

    let centerPoint = window.coordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5))

    centerPoint.tap()
}

This successfully taps the button. However, if I run the same test in landscapeLeft or landscapeRight, the button is not tapped. Printing the coordinate's screenpoint reveals that it is located inside the button's frame in both portrait and landscape modes.

Identical logic is successful for all orientations in Xcode 7 / Swift 2:

func testExample() {

    XCUIDevice.sharedDevice().orientation = .LandscapeLeft

    let window = XCUIApplication().windows.elementBoundByIndex(0)

    let centerPoint = window.coordinateWithNormalizedOffset(CGVectorMake(0.5, 0.5))

    centerPoint.tap()
}

Am I missing something, or is this a legitimate framework bug? Does it have something to do with the transition from CGVectorMake in Swift 2 to CGVector(dx: dy:) in Swift 3?

Answer 1

Same issue — screen points are correct but the actual gesture coordinates are messed up. This workaround did the trick for me:

class SmartXCUICoordinate
{
    let element: XCUIElement
    let normalizedOffset: CGVector

    init(element: XCUIElement, normalizedOffset offset: CGVector) {
        self.element = element
        self.normalizedOffset = offset
    }

    var realCoordinate: XCUICoordinate {
        guard XCUIDevice.shared().orientation.isLandscape else {
            return element.coordinate(withNormalizedOffset: normalizedOffset)
        }

        let app = XCUIApplication()
        _ = app.isHittable // force new UI hierarchy snapshot

        let screenPoint = element.coordinate(withNormalizedOffset: normalizedOffset).screenPoint

        let portraitScreenPoint = XCUIDevice.shared().orientation == .landscapeLeft
            ? CGVector(dx: app.frame.width - screenPoint.y, dy: screenPoint.x)
            : CGVector(dx: screenPoint.y, dy: app.frame.height - screenPoint.x)

        return app
            .coordinate(withNormalizedOffset: CGVector.zero)
            .withOffset(portraitScreenPoint)
    }

    func tap() {
        realCoordinate.tap()  // wrap other XCUICoordinate methods as needed
    }
}

extension XCUIElement
{
    func smartCoordinate(withNormalizedOffset normalizedOffset: CGVector) -> SmartXCUICoordinate {
        return SmartXCUICoordinate(element: self, normalizedOffset: normalizedOffset)
    }
}

Known issue: doesn't work for orientations that your app does not support.

Answer 2

Excellent answer. I had the same coordinate problem testing a [Xcode 8 Swift 3] SpriteKit landscape mode app which I initially incorrectly chalked up to SpriteKit. SmartXCUICoordinate got me back on track.

I added two methods under your comment // wrap other XCUICoordinate methods as needed

//support UILongPressGestureRecognizer
func longPress(){
    realCoordinate.press(forDuration: 2.25)
}
//support drag an element to a second element
//For SpriteKit this drags one SKNode to a second SKNode)
func pressThenDragTo(element:  XCUIElement){
    realCoordinate.press(forDuration: 0.1, thenDragTo: element.smartCoordinate(withNormalizedOffset: CGVector(dx: 0.5, dy: 0.5)).realCoordinate)
}