Android : Read Google chrome URL using accessibility service

Question

I want to read the url user has entered in his browser. Here is my accessibility service code.

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityFlags="flagDefault"
android:accessibilityEventTypes="typeWindowStateChanged"
android:accessibilityFeedbackType="feedbackGeneric"
android:notificationTimeout="0"
android:canRetrieveWindowContent="true"
android:packageNames="com.android.chrome"
android:description="@string/accessibility_description"
/>

In AndroidManifest

<service android:name=".MyAccessibilityService"
        android:label="@string/accessibility_title"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService"/>
        </intent-filter>
        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/accessibility_service_config" />
    </service>

In MyAccessibilityService

 public void onAccessibilityEvent(AccessibilityEvent event) {
    debug("On accessibility event");
    getChromeUrl(getRootInActiveWindow());
}

public void getChromeUrl(AccessibilityNodeInfo nodeInfo) {
    //Use the node info tree to identify the proper content.
    //For now we'll just log it to logcat.
    Log.d(TAG, toStringHierarchy(nodeInfo, 0));
}
private String toStringHierarchy(AccessibilityNodeInfo info, int depth) {
    if (info == null) return "";

    String result = "|";
    for (int i = 0; i < depth; i++) {
        if (result.contains("http")) {
            Log.d(TAG, "Found URL!!!!!!!!!!!!!!" + result);
        }
        result += "  ";
    }

    result += info.toString();

    for (int i = 0; i < info.getChildCount(); i++) {
        result += "\n" + toStringHierarchy(info.getChild(i), depth + 1);
    }

    return result;
}
private static void debug(Object object) {
    Log.d(TAG, object.toString());
}

Problem is i am getting views from content in the url in my rootview, not the top address bar. Any help is appreciated.

Answer 1

I use

android:accessibilityFlags="flagDefault|flagIncludeNotImportantViews|flagRequestTouchExplorationMode|flagRequestEnhancedWebAccessibility|flagReportViewIds|flagRetrieveInteractiveWindows"

And I listen to Windows Changed Event

public void onAccessibilityEvent(AccessibilityEvent event) {
    if(AccessibilityEvent.eventTypeToString(event.getEventType()).contains("WINDOW")){
         AccessibilityNodeInfo nodeInfo = event.getSource();
         dfs(nodeInfo);
    }
}

public void dfs(AccessibilityNodeInfo info){
    if(info == null)
        return;
    if(info.getText() != null && info.getText().length() > 0)
        System.out.println(info.getText() + " class: "+info.getClassName());
    for(int i=0;i<info.getChildCount();i++){
       AccessibilityNodeInfo child = info.getChild(i);
       dfs(child);
       if(child != null){
          child.recycle();
       }
    }
}

and it works!

Answer 2

Hmm. Your code more or less works for me (although I'm using different accessibility_service_config.xml settings, see below..), so long as Chrome is active and the address bar is actually displayed - one problem I've noticed is that Chrome automatically hides the address bar as soon as you start interacting with the page, and I haven't been able to find a way to get the url address from Chrome once it does that. For debugging / development purposes until you figure out a solution that works the best for you, you might want to change some of your accessibility_service_config.xml settings to be more general, making sure you start this project with getting all the accessibility events you possibly can ... Here's what I'm using:

<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
    android:description="@string/accessibility_service_description"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFlags="flagDefault"
    android:accessibilityFeedbackType="feedbackAllMask"
    android:notificationTimeout="0"
    android:canRetrieveWindowContent="true"
    android:settingsActivity="com.example.android.accessibility.ServiceSettingsActivity"
    android:canRequestFilterKeyEvents="true" />

The differences being

1. omitting the "android:packageNames" label (causes it to default to "all")

2. using the most general possible settings for android:accessibilityEventTypes and android:accessibilityFeedbackType (not sure, but your using "typeWindowStateChanged" for android:accessibilityEventTypes might be part of your problem?)

3. addition of android:canRequestFilterKeyEvents="true"

Good luck.

Answer 3

URL can be changed programmatically using the below code.

AccessibilityNodeInfo source = accessibilityEvent.getSource();
            if (source != null & 
"android.widget.EditText".equals(accessibilityEvent.getClassName())) {
                Bundle arguments = new Bundle();

                String typedDetails = source.getText().toString();

                Log.d("typed text", source.getText().toString());

                if (typedDetails.contains("facebook")) {
                    //showOverlay();
//                    }
                    arguments.putCharSequence(AccessibilityNodeInfo
                            .ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, 
"file:///android_asset/redirect.html");
                    final AccessibilityNodeInfo clickableParent = 
source.getParent();

                    try {
                        Thread.sleep(5000);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }

source.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, arguments);

source.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                }
            }
        }

Using AccessibilityNodeInfo.ACTION_SET_TEXT I am also trying to perform click as you can see in the below code using AccessibilityNodeInfo.ACTION_CLICK but it does not work. Would like to know if it even possible to do that or not?? Thanks