问题

Is possible to simulate a touch event in specific screen's coordinates pressing a specific key on a fisical external keyboard (usb via camera connection kit or bluetooth) on ios jailbroken and all the elements that jailbreak involves?

I would use this to press a button in an app (amplitude) with my foot, i want to use a keyboard as a footswitch.

Just for a private use. No Appstore or Cydia. Thanks.

回答1:

You can write scripts on your computer and use your keyboard and mouse to control your iOS device based on the simulate touch library.

iOS13-SimulateTouch is an open-source library that allows you to simulate touch events at the system level. You can write scripts using any programming languages to simulate touch events on your iOS devices either remotely or locally. Please check the source code at [Github] iOS13-SimulateTouch

IOS13-SimulateTouch

iOS 11.0 - 13.6 system-level touch simulation iOS13 simuate touch

Jailbroken device required

Description

This library enables you to simulate touch events on iOS 11.0 - 13.6 with just one line of code! All the source code will be released later.

Features

Multitouch supported (no other library you can find supports multi touching).
Programmable. Control scripts can be programmed with all the programming languages you desire.
Instant controlling supported. The ios device can be controlled with no latency from other devices/computers.
System-level touch simulation (will not inject to any process).

Installation

Open Cydia - Sources - Edit - Add - http://47.114.83.227 ("http" instead of "https"!!! Please double check this.)
Install "ZJXTouchSimulation" tweak
Done

Code Example

Python Version

import socket
import time

# event types
TOUCH_UP = 0
TOUCH_DOWN = 1
TOUCH_MOVE = 2
SET_SCREEN_SIZE = 9

# you can copy and paste this method to your code
def formatSocketData(type, index, x, y):
    return '{}{:02d}{:05d}{:05d}'.format(type, index, int(x*10), int(y*10))

s = socket.socket()
s.connect(("127.0.0.1", 6000))  # connect to the tweak. Replace "127.0.0.1" with the ip address of your device
s.send(("1"+formatSocketData(SET_SCREEN_SIZE, 0, 2048, 2732)).encode())  # tell the tweak that the screen size is 2048x2732 (your screen size might differ). This should be send to the tweak every time you kill the SpringBoard (just send once)
time.sleep(1)  # sleep for 1 sec to get setting size process finished
s.send(("1"+formatSocketData(TOUCH_DOWN, 7, 300, 400)).encode())  # tell the tweak to touch 300x400 on the screen
# IMPORTANT: NOTE the "1" at the head of the data. This indicates the event count and CANNOT BE IGNORED.
s.close()

Actually the touch is performed by only one line:

s.send(("1"+formatSocketData(TOUCH_DOWN, 7, 300, 400)).encode())

Neat and easy.

Perform Touch Move

s.send(("1"+formatSocketData(TOUCH_MOVE, 7, 800, 400)).encode())  # tell the tweak to move our finger "7" to (800, 400)

Perform Touch Up

s.send(("1"+formatSocketData(TOUCH_UP, 7, 800, 400)).encode())  # tell the tweak to touch up our finger "7" at (800, 400)

Combining them

s.send(("1"+formatSocketData(TOUCH_DOWN, 7, 300, 400)).encode())
time.sleep(1)
s.send(("1"+formatSocketData(TOUCH_MOVE, 7, 800, 400)).encode())
time.sleep(1)
s.send(("1"+formatSocketData(TOUCH_UP, 7, 800, 400)).encode())

First the finger touches (300, 400), and then it moves to (800, 400), and then "leaves" the screen. All the touch events are performed with no latency.

Usage

After installation, the tweak will start listening at port 6000.
Use socket to send touch data field to the tweak

data field should always be decimal digits, specified below event data img

NOTE: Usage might be updated in the future. I will update this post but please keep track on github at Usage_iOS13-SimulateTouch

Event Count(1 digit): Specify the count of the single events. If you have multiple events to send at the same time, just increase the event count and append events to the data.

Inside a single event

Type(1 digit): Specify the type of the single event.

Supported event type:

Event: Touch Up. Flag: 0. Description: Specify the event as a touch-up event
Event: Touch Down. Flag: 1. Description: Specify the event as a touch-down event
Event: Touch Move. Flag: 2. Description: Specify the event as a touch-move event (move the finger)
Event: Set Size. Flag: 9. Description: Set screen size (required!! Will be explained below)

Touch Index(2 digits): Apple supports multitouch, so you have to specify the finger index when posting touching events. The range of finger index is 1-20 (0 is reserved, don't use 0 as finger index).

x Coordinate(5 digits): The x coordinate of the place you want to touch. The first 4 digit is for integer part while the last one is for the decimal part. For example, if you want to touch (123.4, 2432.1) on the screen, you should fill "01234" for this.

y Coordinate(5 digits): The y coordinate of the place you want to touch. The first 4 digit is for integer part while the last one is for the decimal part. For example, if you want to touch (123.4, 2432.1) on the screen, you should fill "24321" for this.

Moreover

So if you want to touch down (123.4, 1032.1) with finger "3" on the screen, just connect to the tweak using socket and send "11030123410321".

Digit 0: "1" indicates there is only one event to perform.

Digit 1: "1" indicates event type: TOUCH_DOWN (the flag is 1).

Digit 2-3: "03" indicates this event is performed by finger "3".

Digit 4-8: "01234" indicates the x coordinate 123.4.

Digit 9-13: "10321" indicates the y coordinate 1032.1.

Important Note

The touch coordinate does not depend on the orientation of your device. See picture below to get more information. However you place your device, the click point on the screen will not be changed. coordinate_note img