In iOS Unit Testing – Part 4 – Code Coverage, I discussed how you can evaluate how well your unit tests are exercising your app’s code by using code coverage techniques. With the advent of iOS 7 and Xcode 5, there is an additional step that needs to be done in order to have all that work. An updated version of the final project may be found on GitHub – see the Version8 directory.
As you may recall, the code coverage technique in the previous article uses gcov and lcov-1.10, along with Xcode’s ability to instrument code, to collect coverage information from your unit test run. If you update to Xcode 5 and run your tests using iOS 6.1 or earlier, everything still plays nice. If you run them under iOS 7, however, you will probably get nasty errors like this:
Reading tracefile Coverage.info lcov: ERROR: no valid records found in tracefile Coverage.info Reading tracefile Coverage.info lcov: ERROR: no valid records found in tracefile Coverage.info Reading tracefile Coverage.info lcov: ERROR: no valid records found in tracefile Coverage.info Reading data file Coverage.info genhtml: ERROR: no valid records found in tracefile Coverage.info
The problem is that the .gcda files with coverage information don’t get generated as a result of your unit test run. The underlying reason for this has to do with changes in the way that your application gets terminated at the end of the run – the files are normally written out as the app is shut down.
Fortunately, there’s a way around this. (If there wasn’t, I wouldn’t be writing this piece.) You can install a “test observer” class, and have that class force the gcov data to be flushed out.
In your unit test bundle, add a class that looks like this:
#import <SenTestingKit/SenTestingKit.h>
@interface SBTestObserver : SenTestLog
@end
static id mainSuite = nil;
@implementation SBTestObserver
+ (void)initialize
{
[[NSUserDefaults standardUserDefaults] setValue:@"SBTestObserver" forKey:SenTestObserverClassKey];
[super initialize];
}
+ (void)testSuiteDidStart:(NSNotification*)notification
{
[super testSuiteDidStart:notification];
SenTestSuiteRun* suite = notification.object;
if (mainSuite == nil)
{
mainSuite = suite;
}
}
+ (void)testSuiteDidStop:(NSNotification*)notification
{
[super testSuiteDidStop:notification];
SenTestSuiteRun* suite = notification.object;
if (mainSuite == suite)
{
UIApplication* application = [UIApplication sharedApplication];
[application.delegate applicationWillTerminate:application];
}
}
@end
The name of the class doesn’t matter, but it is important that you modify line 11 (highlighted above) so that it matches the class name. Once this is done, the testSuiteDidStart and testSuiteDidStop notifications will get called as part of the test cycle.
As you can see, testSuiteDidStop is calling the application delegate’s applicationWillTerminate method. Here is where the second part comes in – in my application, this method is written as follows:
extern void __gcov_flush(void);
- (void)applicationWillTerminate:(UIApplication *)application
{
#ifdef DEBUG
__gcov_flush(); // flush code coverage data
#endif
}
As you can see, it calls __gcov_flush. This will force all the code coverage information to be flushed out at this point. With these two parts in place, the coverage information gets written out as the test suite is finishing up, as opposed to when the process being tested (your app) terminates. As a result, it no longer matters exactly how the app process gets shut down – effectively, we’ve built the “flush” into the test cycle itself, so that it’s written before OCUnit gets done. (I presume this would also work with the new XCTest stuff – I just haven’t tried it yet.)
It should be noted that the call to __gcov_flush must be made from within the actual application code. If you call it from the test observer, it doesn’t work, because the coverage information is accumulated in the app, not in your test suite. Remember that since Xcode 4, your test code is in a completely separate bundle, which gets “grafted onto” your application when the test suite is executed.
Of course, your applicationWillTerminate may be very different. It really doesn’t matter exactly where the call to __gcov_flush gets placed, as long as it’s in your app bundle and gets called as the test suite is shutting down. applicationWillTerminate is simply a convenient and logical place to put it.
This brings us to another item that I’ll pass along to you. Recently, I had a class that just wouldn’t seem to save code coverage information. I could confirm with breakpoints that the methods on the class were actually getting called, but when I ran the code coverage, all the lines were listed as uncovered. After much hair pulling (a bad thing, since I don’t have much to spare), I finally figured out what was going on. When a added the new class to the project, I had somehow reverted back to unit-tests-in-Xcode-3 mentality, and had added the class to both the app target and also to the unit test target. Those of you who’ve been doing this long enough remember that, back before Xcode 4, you put your code to be tested into both targets and your test code into only the test target, because back then the test target was a standalone executable containing both the tests and the objects being tested. As a result, my unit tests were working on the version in the test bundle, not the one in the app bundle and, of course, no coverage information was being generated in the test bundle. As soon as I did the correct thing – having the app code only belong to the app bundle – it all played nice.
So don’t repeat my mistake – remember that in the “modern Xcode world,” your app code goes in the app target, test code in the test target, and never the two shall share…