Android Gradle Running Tests on Non-Debug Builds

Question

I have a project with three different build types: debug, beta, and release. My test package is always created for debug builds, but QA uses the beta build and we want QA to run these tests on their vast array of devices.

I'm trying to create a testing apk for QA that is signed by the same key as the beta build. Looking through the Android-Gradle documentation, I don't see anything telling me that I can't do this, but I don't see anyway to configure this. Is there anyway I can configure which keystore is used when assembling a test apk? Or is there a way to create an unsigned test apk?

Answer 1

This is an incomplete answer to your question in that it documents what you can't do, but the connectedAndroidTest task, which is what runs the androidTest tests in your project, is hardcoded to run against the debug build type, and I don't see a way to point it at a different build type.

Taking the advice from Is there a way to list task dependencies in Gradle? and examining the task dependency tree, if you run:

./gradlew tasks --all

you get this in your output:

Verification tasks
------------------
app:check - Runs all checks. [app:lint]
app:connectedAndroidTest - Installs and runs the tests for Build 'debug' on connected devices. [app:assembleDebug, app:assembleDebugTest]
app:connectedCheck - Runs all device checks on currently connected devices. [app:connectedAndroidTest]
app:deviceCheck - Runs all device checks using Device Providers and Test Servers.

The documentation for the connectedAndroidTest task claims it runs tests against debug, and the task dependencies (which you see with the -all flag) confirm that the task depends on assembleDebug.

Adding additional build types and flavors doesn't seem to affect the dependency on the built-in debug type.

It's possible that with greater Gradle-fu than mine, you could rewire the tasks to make the tests depend on a different build type, but doing this is likely to be fragile since it's bound to depend on things that aren't supported API in the Android Gradle plugin.

To answer your question most directly, though, if all you want is to run tests against a build with a different certificate, you could change the signing config on your debug build to use the beta certificate:

android {
    signingConfigs {
        beta {
            keyAlias 'key'
            keyPassword 'password'
            storeFile file('/path/to/beta_keystore.jks')
            storePassword 'password'
        }
    }
    buildTypes {
        debug {
            signingConfig signingConfigs.beta
        }
        beta {
            signingConfig signingConfigs.beta
        }
    }
}

I tested it and I am able to run androidTest targets against debug builds that use a custom keystore in this way. However, I doubt this solves your problem, because I suspect you want to run your tests against the beta build, not a debug build with the beta certificate.

Answer 2

You can now point this to a different target, I don't know when this happened, but from the docs:

Currently only one Build Type is tested. By default it is the debug Build Type, but this can be reconfigured with:

android {
    ...
    testBuildType "staging" 
}

Answer 3

To add a testing source set for your build variant, follow these steps:

• In the Project window on the left, click the drop-down menu and select the Project view.
• Within the appropriate module folder, right-click the src folder and click New > Directory.
• For the directory name, enter "androidTestVariantName." For example, if you have a build variant called "MyFlavor" then the directory name shoulbe "androidTestMyFlavor." Then click OK.
• Right-click on the new directory and click New > Directory. Enter "java" as the directory name, and then click OK.

Now you can add tests to this new source set by following the steps above to add a new test. When you reach the Choose Destination Directory dialog, select the new variant test source set.

The instrumented tests in src/androidTest/ source set are shared by all build variants. When building a test APK for the "MyFlavor" variant of your app, Gradle combines both the src/androidTest/ and src/androidTestMyFlavor/ source sets.

Another way is to put following line your in default config.

Currently only one Build Type is tested. By default it is the debug Build Type, but this can be reconfigured with:

android {
    ...
    testBuildType "staging"
}