OCUnit tests fail from the command line but work in Xcode when using Keychain Services

Question

I'm using SFHFKeychainUtils to use Keychain Services in my app. I've written some OCUnit tests that verify the funcionality of this code. Everything works fine when I run the unit tests from Xcode on the iOS simulator or my device. However now I'm trying to set up a CI server and the test is failing when it is run via the command line with error code -25291. Looking that up on Apple's documentation tells me: No trust results are available (errSecNotAvailable). I've linked the Security.framework to my unit test project, it seems like from what I have read on the web this is all I should need to get this working. Here is the command I am invoking in the console:

/usr/bin/xcodebuild -target [Test_Target] -sdk /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator5.1.sdk/ -configuration Debug

Does anyone have any experience or suggestions for getting unit testing and Keychain Services to play nicely together from the command line?

Answer 1

I ran into the same issue, and the solution for me was to make sure the simulator was running before starting any test. I did that using AppleScript in a Run Script build phase in Xcode, and essentially the same thing on the CI server. Here is the shell script that will open the simulator:

exec osascript <<EOF

tell application "iOS Simulator"

activate

end tell

The security/keychain services issue that causes this is apparently a known issue, though I don't yet have the radar that tracks it. If you're using Jenkins, put the above script in a Execute Shell phase before your Xcode build phase. If you're controlling this through Xcode itself, put it in a Run Script build phase before the RunUnitTests Run Script build phase. Hope that solves your issue!

Answer 2

I wasn't able to figure out why keychain access fails when OCUnit tests are run from the command line.

In order to proceed with my testing I added a hacktastic category to my unit test target:

//
//  SFHFKeychainUtils+UnitTestHacks.m
//  CB30
// 
// GRRR!!! http://stackoverflow.com/questions/9996578/ocunit-tests-fail-from-the-command-line-but-work-in-xcode-when-using-keychain-se
//
//  Created by Joshua Vickery on 5/14/12.
//

#import "SFHFKeychainUtils+UnitTestHacks.h"

static NSMutableDictionary *fakeKeyChainHolder;

@implementation SFHFKeychainUtils (UnitTestHacks)

+ (NSMutableDictionary *)fakeKeyChainForServiceName:(NSString *)serviceName {
    if (nil == fakeKeyChainHolder) {
        fakeKeyChainHolder = [NSMutableDictionary dictionary];
    }

    NSMutableDictionary *fakeKeyChain = [fakeKeyChainHolder objectForKey:serviceName];
    if (nil == fakeKeyChain) {
        fakeKeyChain = [NSMutableDictionary dictionary];
        [fakeKeyChainHolder setObject:fakeKeyChain forKey:serviceName];
    }
    return fakeKeyChain;
}

#pragma clang diagnostic push
#pragma clang diagnostic ignored "-Wobjc-protocol-method-implementation"

+ (BOOL) deleteItemForUsername: (NSString *) username andServiceName: (NSString *) serviceName error: (NSError **) error 
{
    [[self fakeKeyChainForServiceName:serviceName] removeObjectForKey:username];
    return YES;
}


+ (BOOL) storeUsername: (NSString *) username andPassword: (NSString *) password forServiceName: (NSString *) serviceName updateExisting: (BOOL) updateExisting error: (NSError **) error 
{
    [[self fakeKeyChainForServiceName:serviceName] setObject:password forKey:username];
    return YES;
}

+ (NSString *) getPasswordForUsername: (NSString *) username andServiceName: (NSString *) serviceName error: (NSError **) error 
{
    return [[self fakeKeyChainForServiceName:serviceName] objectForKey:username];
}

#pragma clang diagnostic pop

@end

Please note that this is not a good solution but a work-around to get unit tests working until a better solution is found.

Answer 3

I am experiencing a similar problem. From my research this might come down to the simulator version that's running when the tests are running. Unit testing keychain with iphone simulator 6.0 up will always make my tests fail, if running from the command line. Change it to any other version (4.3, 5.0, 5.1) and they pass. Always good from XCode with any version though.

Maybe this is a command line tools problem which is not setting up some necessary flags before running tests.

I opened up a new question here with a small test case: Keychain Services API fails with errSecNotAvailable in iphonesimulator 6.0