How to pass POINT structure to ElementFromPoint method in Python?

Question

I\'m trying to use method IUIAutomation::ElementFromPoint in Python using comtypes package. There are many examples how to use it in C++, b

Answer 1

You can reuse existing POINT structure definition directly, like this:

import comtypes
from comtypes import *
from comtypes.client import *

comtypes.client.GetModule('UIAutomationCore.dll')
from comtypes.gen.UIAutomationClient import *

# get IUIAutomation interface
uia = CreateObject(CUIAutomation._reg_clsid_, interface=IUIAutomation)

# import tagPOINT from wintypes
from ctypes.wintypes import tagPOINT
point = tagPOINT(10, 10)
element = uia.ElementFromPoint(point)

rc = element.currentBoundingRectangle # of type ctypes.wintypes.RECT
print("Element bounds left:", rc.left, "right:", rc.right, "top:", rc.top, "bottom:", rc.bottom)

To determine what's the expected type for ElementFromPoint, you can just go to your python setup directory (for me it was C:\Users\<user>\AppData\Local\Programs\Python\Python36\Lib\site-packages\comtypes\gen) and check the files in there. It should contains files automatically generated by comtypes, including the one for UIAutomationCore.dll. The interesting file name starts with _944DE083_8FB8_45CF_BCB7_C477ACB2F897 (the COM type lib's GUID).

The file contains this:

COMMETHOD([], HRESULT, 'ElementFromPoint',
          ( ['in'], tagPOINT, 'pt' ),

This tells you that it expects a tagPOINT type. And this type is defined a the beginning of the file like this:

from ctypes.wintypes import tagPOINT

It's named tagPOINT because that's how it's defined in original Windows header.