Python Mouse Movement Emulation in Games

问题

I'm looking into using the Kinect to play video games. This requires the emulation of both keyboard and mouse events. I've figured out that keyboard events with ctypes' SendInput() works in both Skyrim and Minecraft. I've also found that ctypes' mouse_event() works in both Skyrim and Minecraft for emulating mouse buttons. Those are the solutions I've found so far that seem to work for both Skyrim and Minecraft.

The main issue I'm having is with moving the player's camera around within Skyrim. I'm able to move the camera around in Minecraft using ctypes' SetUserPos() and also ctypes' mouse_event(). But neither solutions work for Skyrim. I've also tried using SendInput() to move the player's camera in Skyrim, but every time the cursor gets moved with SendInput() (even outside of Skyrim) my monitor goes blank for a moment and the camera still doesn't move (I have no idea why something like that is happening...)

So is there anyway that I could possibly be able to emulate the camera movement within Skyrim or other games that probably handle their mouse inputs similarly? I'm willing to use C/C++ for this task, but straight Python would be more preferable. Thanks for any and all suggestions you may have!

I'm also going to leave the code I used to make my monitor go blank. I'm thinking SendInput() probably won't work for camera movement within Skyrim, but maybe I just did something terribly wrong. (I also got most of the below code from a multitude of threads online.)

import ctypes
import time

# C struct redefinitions
PUL = ctypes.POINTER(ctypes.c_ulong)

class KeyBdInput(ctypes.Structure):
    _fields_ = [("wVk", ctypes.c_ushort),
                ("wScan", ctypes.c_ushort),
                ("dwFlags", ctypes.c_ulong),
                ("time", ctypes.c_ulong),
                ("dwExtraInfo", PUL)]


class HardwareInput(ctypes.Structure):
    _fields_ = [("uMsg", ctypes.c_ulong),
                ("wParamL", ctypes.c_short),
                ("wParamH", ctypes.c_ushort)]


class MouseInput(ctypes.Structure):
    _fields_ = [("dx", ctypes.c_long),
                ("dy", ctypes.c_long),
                ("mouseData", ctypes.c_ulong),
                ("dwFlags", ctypes.c_ulong),
                ("time", ctypes.c_ulong),
                ("dwExtraInfo", PUL)]


class POINT(ctypes.Structure):
        _fields_ = [("x", ctypes.c_ulong),
                    ("y", ctypes.c_ulong)]


class Input_I(ctypes.Union):
    _fields_ = [("ki", KeyBdInput),
                 ("mi", MouseInput),
                 ("hi", HardwareInput)]


class Input(ctypes.Structure):
    _fields_ = [("type", ctypes.c_ulong),
                ("ii", Input_I)]

# Actuals Functions


def PressKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput(0, hexKeyCode, 0x0008, 0, ctypes.pointer(extra))
    x = Input(ctypes.c_ulong(1), ii_)
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))


def ReleaseKey(hexKeyCode):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    ii_.ki = KeyBdInput(0, hexKeyCode, 0x0008 | 0x0002, 0, ctypes.pointer(extra))  # lint:ok
    x = Input(ctypes.c_ulong(1), ii_)
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

def MoveMouse(x, y):
    extra = ctypes.c_ulong(0)
    ii_ = Input_I()
    x = int(x*(65536/ctypes.windll.user32.GetSystemMetrics(0))+1)
    y = int(y*(65536/ctypes.windll.user32.GetSystemMetrics(1))+1)
    ii_.mi = MouseInput(x, y, 0, 0x0001 | 0x8000, 1, ctypes.pointer(extra))
    x = Input(ctypes.c_ulong(0), ii_)
    ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))

def main():
    mouse = Mouse()
    time.sleep(3)
    MoveMouse(100, 100)

main()

回答1:

You can install the PyAutoGUI GUI automation module from PyPI (run pip install pyautogui) and then call the pyautogui.moveTo() to click on a certain X and Y coordinates of the screen:

>>> import pyautogui
>>> pyautogui.moveTo(50, 100)

Behind the scenes on Windows, it makes similar ctypes code you are using, so it might work to control the camera in a game like Skyrim.

PyAutoGUI works on Windows, Mac, and Linux, and on Python 2 and 3. It also can emulate the keyboard, do mouse drags, take screenshots, and do simple image recognition of the screenshots. Full docs are at https://pyautogui.readthedocs.org/

来源：https://stackoverflow.com/questions/27674827/python-mouse-movement-emulation-in-games