Occasional errors when running OCUnit application test suite on device

Question

I'm having some problems with my OCUnit test suite, when running application tests (i.e. on the device). Occasionally, I get a tricky error that seems to be thrown from the OCUnit classes. I've been trying to look through the files added to the test suites, but can't get my head around it.

I've followed the Apple documentation and looked at other tutorials when setting up my test targets, resulting in the following targets (and indented dependencies):

LogicTests

MyApp LogicTests

ApplicationTests MyApp

MyAppTesting ApplicationTests

The last (MyAppTesting) is the one I run on the device to start the test suite.

And as mentioned, the tests fail to start sometimes throwing me the following output in the console:

2010-11-17 12:02:48.537 VCCTesting[12018:307] *** Assertion failure in -[SenTestClassEnumerator init], /SourceCache/OCUnit/OCUnit-1685/SourceCode/SenTestingKit/SenTestClassEnumerator.m:137

2010-11-17 12:02:48.556 VCCTesting[12018:307] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'objc_getClassList returned more classes than it should have.'
*** Call stack at first throw:
(
0   CoreFoundation                      0x30897ed3 __exceptionPreprocess + 114
1   libobjc.A.dylib                     0x3002f811 objc_exception_throw + 24
2   CoreFoundation                      0x30897d15 +[NSException raise:format:arguments:] + 68
3   Foundation                          0x349f932f -[NSAssertionHandler handleFailureInMethod:object:file:lineNumber:description:] + 62
4   SenTestingKit                       0x20107c29 -[SenTestClassEnumerator init] + 292
5   SenTestingKit                       0x201079fd +[SenTestClassEnumerator classEnumerator] + 24
6   SenTestingKit                       0x2010777d +[NSObject(SenTestRuntimeUtilities) senAllSubclasses] + 92
7   SenTestingKit                       0x201067b5 +[SenTestSuite updateCache] + 28
8   SenTestingKit                       0x20106753 +[SenTestSuite suiteForBundleCache] + 54
9   SenTestingKit                       0x201065db +[SenTestSuite structuredTests] + 14
10  SenTestingKit                       0x201065a1 +[SenTestSuite allTests] + 40
11  SenTestingKit                       0x20106573 +[SenTestSuite defaultTestSuite] + 14
12  SenTestingKit                       0x201057a3 +[SenTestProbe specifiedTestSuite] + 106
13  SenTestingKit                       0x20105d83 +[SenTestProbe runTests:] + 94
14  Foundation                          0x349c7e8d __NSFireDelayedPerform + 368
15  CoreFoundation                      0x3084e7fb __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 14
16  CoreFoundation                      0x3084e2ad __CFRunLoopDoTimer + 860
17  CoreFoundation                      0x3081f7a5 __CFRunLoopRun + 1088
18  CoreFoundation                      0x3081f277 CFRunLoopRunSpecific + 230
19  CoreFoundation                      0x3081f17f CFRunLoopRunInMode + 58
20  GraphicsServices                    0x31e445f3 GSEventRunModal + 114
21  GraphicsServices                    0x31e4469f GSEventRun + 62
22  UIKit                               0x31e51123 -[UIApplication _run] + 402
23  UIKit                               0x31e4f12f UIApplicationMain + 670
24  VCCTesting                          0x000234ff main + 70
25  VCCTesting                          0x00002538 start + 40
)
terminate called after throwing an instance of 'NSException'

Does anyone know what could be wrong?

Thankful for any help, Kristofer

P.S. I've posted this on the App dev forums, but haven't had any luck with it yet: https://devforums.apple.com/message/333890

Answer 1

Do you have NSZombieEnabled? I had this issue when I had NSZombieEnabled set to YES in my arguments for my unit test launch target. I've tried everything (cleaning/quitting XCode/restting simulator content). The only solution so far was to disable zombie in my unit test launch

Answer 2

I think I've just figured out why I was getting this error as well -- perhaps the same thing is happening to you:

I have two products set up, one for internal beta testing (complete with a preference pane) and one for app store release. Both product targets were set up with the same Product Name (PRODUCT_NAME in build settings).

This worked fine in XCode 3.x, but XCode 4 was somehow trying to build the store product each time I built the beta product. I think this is due to the change in where XCode puts its derived files -- XCode 4 was storing the beta and store products at the same path, so when the beta product was built, the store output file changed and needed to be rebuilt...

Anyway... changing the beta Product Name has fixed temporarily fixed the issue... I've also changed the bundle identifier in the Info.plist so the two products have different identifiers. Now when I build one product, the other doesn't get built at the same time, but I still get the crash when the tests run :-(

My only idea now is to totally recreate the project from fresh, copying across the settings and the files to see if XCode 4 does something different. That and filing a support incident with Apple...

Cheers,

Adam

Answer 3

I am not 100% sure yet, but it seems this issue can be solved by creating different scheme for each test target. I had this error a lot this afternoon but it hasn't happened for awhile after I created separate schemes for different test targets.

Answer 4

Just for anyone that might run into this issue like I have:

KIFTester loaded
Assertion failure in -[SenTestClassEnumerator init], /SourceCache/OCUnit/OCUnit-5315/SourceCode/SenTestingKit/SenTestClassEnumerator.m:136
Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'objc_getClassList returned more classes than it should have.'

The problem turned out to be GoogleAnalytics 3.0 Updated to 3.0.3 but still the same problem with either of these libraries libGoogleAnalytics_debug.a libGoogleAnalyticsServices.a

To go around it I just don't include them for testing.