I am executing CTS on Jacinto 6 Evaluation Module (ti-jacinto6evm) and I'm encountering a number of test case failures that I don't understand.
I started by building both AOSP and CTS. Both builds were just fine. I can flash my test hardware (ti-jacinto6evm) and then I followed the instructions for setting up CTS. I have run CTS for more then 10 times on the same device and every time I got different results. The ti-jacinto6 device randomly gets hanged during execution of the test cases.
Most of the time target gets hanged and it show following error:
Reason: 'Failed to receive adb shell test output within 600000 ms. Test may have timed out, or adb connection to device became unresponsive'. Check device logcat for details Device 170090035a700002 shell is unresponsive 05-30 04:52:21 W/TestInvocation: Invocation did not complete due to device 170090035a700002 becoming not available. Reason: Could not find device 170090035a700002 on the below test cases my target hangs:
CtsPreference2TestCases CtsUiHostTestCases CtsServicesHostTestCases CtsTrustedVoiceHostTestCases CtsTransitionTestCases CtsAppTestCases CtsGraphicsTestCases CtsCameraTestCases CtsWebkitTestCases CtsFragmentTestCases CtsViewTestCases So I just excluded those test cases from the CTS and again ran CTS with the following command:
run cts --skip-preconditions --exclude-filter CtsPreference2TestCases --exclude-filter CtsServicesHostTestCases --exclude-filter CtsUiHostTestCases --exclude-filter CtsTrustedVoiceHostTestCases --exclude-filter CtsAppTestCases --exclude-filter CtsGraphicsTestCases --exclude-filter CtsTransitionTestCases --exclude-filter CtsCameraTestCases --exclude-filter CtsWebkitTestCases --exclude-filter CtsFragmentTestCases --plan cts Problem 1
I am facing a problem where some test cases are running properly for the first time, but when I run CTS for the second time, they fail some passed test case(s).
1st iteration. On this iteration 166 modules passed:
2nd iteration. On this iteration 91 modules passed:
Problem 2
When CTS gets stuck on some testcases it shows a TimeoutException:
com.android.ddmlib.TimeoutException at com.android.ddmlib.AdbHelper.read(AdbHelper.java:767) at com.android.ddmlib.AdbHelper.read(AdbHelper.java:736) at com.android.ddmlib.AdbHelper.readAdbResponse(AdbHelper.java:222) at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:456) at com.android.ddmlib.AdbHelper.executeRemoteCommand(AdbHelper.java:382) at com.android.ddmlib.Device.executeShellCommand(Device.java:617) at com.android.tradefed.device.NativeDeviceStateMonitor.waitForDeviceShell(NativeDeviceStateMonitor.java:170) at com.android.tradefed.device.WaitDeviceRecovery.recoverDevice(WaitDeviceRecovery.java:142) at com.android.tradefed.device.NativeDevice.recoverDevice(NativeDevice.java:1720) at com.android.tradefed.device.NativeDevice.performDeviceAction(NativeDevice.java:1661) at com.android.tradefed.device.NativeDevice.runInstrumentationTests(NativeDevice.java:615) at com.android.tradefed.device.NativeDevice.runInstrumentationTests(NativeDevice.java:698) at com.android.tradefed.testtype.InstrumentationTest.runWithRerun(InstrumentationTest.java:797) at com.android.tradefed.testtype.InstrumentationTest.doTestRun(InstrumentationTest.java:740) at com.android.tradefed.testtype.InstrumentationTest.run(InstrumentationTest.java:643) at com.android.tradefed.testtype.AndroidJUnitTest.run(AndroidJUnitTest.java:233) at com.android.compatibility.common.tradefed.testtype.ModuleDef.run(ModuleDef.java:250) at com.android.compatibility.common.tradefed.testtype.CompatibilityTest.run(CompatibilityTest.java:506) at com.android.tradefed.invoker.TestInvocation.runTests(TestInvocation.java:761) at com.android.tradefed.invoker.TestInvocation.prepareAndRun(TestInvocation.java:446) at com.android.tradefed.invoker.TestInvocation.performInvocation(TestInvocation.java:300) at com.android.tradefed.invoker.TestInvocation.invoke(TestInvocation.java:886) at com.android.tradefed.command.CommandScheduler$InvocationThread.run(CommandScheduler.java:567) What is the reason behind this failure?