pywin32 / pywinauto not working properly in remote desktop when it is minimized

Question

I have a Jenkins pipeline that executes a program in a remote server that uses pywin to manipulate an application for functional tests.

My application works great while I have the remote desktop oppened but when I close the remote desktop and run it from Jenkins, the app gets lost.

What I do is to open the app and send an enter key.

This is my app:

os.startfile("C:\\Program Files (x86)\\SAP\\FrontEnd\\SAPgui\\saplogon.exe")

time.sleep(5)
handle = win32gui.FindWindow(0, "SAP Logon 740")  

keyboard = Controller()
keyboard.press(Key.enter)

So I tried to add focus to the app to force the focus without success:

os.startfile("C:\\Program Files (x86)\\SAP\\FrontEnd\\SAPgui\\saplogon.exe")

time.sleep(5)
handle = win32gui.FindWindow(0, "SAP Logon 740")  

win32gui.ShowWindow(handle, 5)           
win32gui.SetForegroundWindow(handle)

keyboard = Controller()
keyboard.press(Key.enter)

I changed the key presses to this with the same result:

shell = win32com.client.Dispatch("WScript.Shell")
shell.SendKeys('{ENTER}')

I tried changing to pywinauto trying to do a click on a button instead of sending the enter but I found more problems, as pywinauto is not recognizing my app titles:

app = Application().start("C:\\Program Files (x86)\\SAP\\FrontEnd\\SAPgui\\saplogon.exe")

app["SAP Logon 740"] # this doesn't work
app.top_window_()    # this doesn't work either

handle = win32gui.FindWindow(0, "SAP Logon 740")  
sapApp = app.window_(handle = handle)            #Finally this works but...

sapApp["Log &On"].click()                        # This doesn't work
sapApp.log_on.Click()                            # This doesn't work

I get this exception:

ctypes.ArgumentError: argument 2: <class 'TypeError'>: expected LP_c_ulong instance instead of pointer to c_long

I know that's the name because I tried:

print(sapApp.descendants(control_type="MenuBar"))

And got this results:

[<win32_controls.ButtonWrapper - 'Log &On', Button, 14221798>, ...]

So I know that's the name of the button but haven't been able to click on it.

I also tried setting focus to the app, with the same result:

sapApp.SetFocus()

With the remote desktop oppened everything works fine but it its closed my app doesn't get the enter

So has anyone had this problem before? I'm running out of ideas what else can I try?

Thanks

EDIT:

This is the full trace for the ctypes error:

File "e:\Jenkins\workspace\my-project\scripts\test_pywin.py", line 23, in <module> sapApp.log_on.Click()
File "E:\Python_V365\lib\site-packages\pywinauto\controls\hwndwrapper.py", line 725, in click self.verify_actionable()
File "E:\Python_V365\lib\site-packages\pywinauto\base_wrapper.py", line 591, in verify_actionable self.wait_for_idle()
File "E:\Python_V365\lib\site-packages\pywinauto\controls\hwndwrapper.py", line 710, in wait_for_idle win32functions.WaitGuiThreadIdle(self)
File "E:\Python_V365\lib\site-packages\pywinauto\win32functions.py", line 283, in WaitGuiThreadIdle GetWindowThreadProcessId(handle, ctypes.byref(process_id))
ctypes.ArgumentError: argument 2: <class 'TypeError'>: expected LP_c_ulong instance instead of pointer to c_long

Also tried (backend="uia") to start the app with the same results:

app = Application(backend="uia").start("C:\\Program Files (x86)\\SAP\\FrontEnd\\SAPgui\\saplogon.exe")

And trying the connect after starting the program gives me this error:

app = Application().connect(title="SAP Logon 740", timeout=10)

File "e:\Jenkins\workspace\tacion_BehaveImplementation-637TPHZXXSFG4MVWWWJCBSJOWSAVPZMPOYFKFKNYKRT5XRBIZFBQ\scripts\test_pywin.py", line 12, in <module> app = Application().connect(title="SAP Logon 740", timeout=10)
File "E:\Python_V365\lib\site-packages\pywinauto\application.py", line 944, in connect self.process = findwindows.find_element(**kwargs).process_id
File "E:\Python_V365\lib\site-packages\pywinauto\findwindows.py", line 84, in find_element elements = find_elements(**kwargs)
TypeError: find_elements() got an unexpected keyword argument 'timeout'

This finally works:

app = Application().connect(title="SAP Logon 740", backend="uia")
sapApp = app["SAP Logon 740"]

But found out the problem is actually that I need to wait for the app to fully load so this works too:

app = Application().start("C:\\Program Files (x86)\\SAP\\FrontEnd\\SAPgui\\saplogon.exe")
time.sleep(5)
app["SAP Logon 740"]

But still the same problem with the button

Answer 1

UPD: all practices listed below are compiled into Remote Execution Guide which may be more up-to-date.

The problem with Remote Desktop doesn't depend on the tool. The RDP itself doesn't keep GUI context when it's minimized or disconnected (the same effect takes place when the OS is locked). But the symptoms usually involve .click_input() and .type_keys() / keyboard.SendKeys() methods that are not working without GUI context.

The workarounds for minimized RDP problem:

1. Switch RDP to windowed mode (non full screen), run the script there and quickly switch to local machine. Now you can work normally. This is the simplest way for manual run.

2. Install VNC server software on remote machine (I used TightVNC) and VNC client (also TightVNC) on local one. You may require updating video card drivers on remote machine if you see black screen. You also must reboot remote host if RDP was used at least once. The main benefit: you can even disconnect from remote host but TightVNC will always keep GUI context. This is more suitable for automatic runs (Jenkins agent must run in this active desktop, it can't be run as a service at all). Another virtual desktop environment like Citrix might be OK for this purpose, but I have no personal experience with that.

3. RDP (mstsc command) has some parameter to unbind virtual remote desktop from current connection (don't remember right now). Additional tools for automatic remote run may include psexec or Ansible with psexec plugin.

---

If the main window is not found for this application (process PID) maybe saplogon.exe spawns another process with the target window. Then you have to do app = Application().connect(title="SAP Logon 740", timeout=10) to bind with correct process ID. This is pretty common problem for such launchers.

---

ctypes.ArgumentError is more interesting. Please provide the full traceback of the error. I suspect this can be fixed on pywinauto side. This kind of error may happen because other Python libraries use ctypes incorrect way, but it can be workarounded probably.

---

If LogOn button can't be found using default "win32" backend, you may try Application(backend="uia").connect(...). The difference is explained in the Getting Started Guide.