macOS accessibility API on WebKit applications with AXTextMarker

Question

I need to access data from webkit applications such as Safari, Mail and maybe others. I can see in the Accessibility Inspector there is :AXTextMarker

Answer 1

Tips and tricks:

Ask focussedElement for its supported attributes. Use the functions:

AXError AXUIElementCopyAttributeNames(AXUIElementRef element, CFArrayRef  _Nullable *names);

Returns a list of all the attributes supported by the specified accessibility object.

and

AXError AXUIElementCopyParameterizedAttributeNames(AXUIElementRef element, CFArrayRef  _Nullable *names);

Returns a list of all the parameterized attributes supported by the specified accessibility object.

Most undocumented attributes are self explanatory.

For example get the selected text as attributed string:

CFTypeRef markerRange = NULL;
AXError error = AXUIElementCopyAttributeValue(focussedElement, (CFStringRef)@"AXSelectedTextMarkerRange", &markerRange);
CFTypeRef result = NULL;
error = AXUIElementCopyParameterizedAttributeValue(focussedElement, (CFStringRef)@"AXAttributedStringForTextMarkerRange", markerRange, &result);

Answer 2

Accessibility apps need to be added in System Preferences in the following section:

System Preferences -> Security & Privacy -> Privacy tab -> Accessibility (on the left)

Note that during development you need to include Xcode in this list if you are developing/running within Xcode.

I didn't test out your exact code, but I tested some comparable swift code and that worked fine provided Xcode was added in that spot in system preferences.

Answer 3

Try this, I haven't tested this but I found some research on the API and I think this may solve your problem.

It seems for the accessibility API to fully work the application needs to be trusted or basically have root access. Is the application running in root?

I found some apple script from a past project. seemed relevant

NSDictionary *errorInfo = [NSDictionary new];
NSString *script =  [NSString stringWithFormat:@"do shell script \"%@ %@\" with administrator privileges", @"/usr/sbin/lsof",@"-c filecoord"];
NSLog(@"Running... %@",script);
NSAppleScript *appleScript = [[NSAppleScript new] initWithSource:script];
NSAppleEventDescriptor * eventResult = [appleScript executeAndReturnError:&errorInfo];

This apple script isn't excatily what you need but its a start. I'm not sure how to fully request root access to an application from inside the application its self, But for testing, you can always just run your application as root and continue your debugging

Please let me know if this helped you in any way or not