NSOperation and NSURLConnection mystified

Question

I am trying to download multiple images from some server using NSOperation and NSOperationQueue. My main question is what is the difference between the code snippet below, and

Answer 1

I'm going to tackle these in reverse order. You ask:

So this brings me to my second question, why does [queue setMaxConcurrentOperationCount:1] have such a big affect in my code below? From the documentation, I thought that leaving the maxConcurrentOperationCount at its default value was fine, and that just tells the queue to decide what the best value should be based on certain factors.

With NSURLConnection, you cannot have more than four or five connections downloading concurrently. Thus, if you don't set maxConcurrentOperationCount, the operation queue doesn't know you're dealing with NSURLConnection and therefore when you add 300 NSOperation objects to your queue, the queue will try to start a very large number of them (64-ish, I think) concurrently. But since only 4 or 5 NSURLConnection requests can run concurrently, the rest of them that were started by the queue will wait until one of the four or five possible connections are available, and with so many download requests, it's quite likely that many of them will time out and fail.

By using maxConcurrentOperationCount of 1, that applies a rather heavy-handed solution to this problem, only running one at a time. I'd suggest a compromise, namely a maxConcurrentOperationCount of 4, which enjoys a degree of concurrency (and huge performance gain), but not so many that we risk having connections time out and fail.

Going back to Dave Drubin's NSOperation, his is great improvement over your synchronousRequest wrapped in an operation. Having said that, he's neglected to address a fairly basic feature of concurrent requests, namely cancelation. You should include a check to see if the operation has been canceled, and if so, cancel the connection:

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    if ([self isCancelled]) {
        [connection cancel];
        [self finish];
        return;
    }

    [_data appendData:data];
}

Likewise, when he should be doing that in the start method, too.

- (void)start
{
    // The Apple docs say "Always check for cancellation before launching the task."

    if ([self isCancelled]) {
        [self willChangeValueForKey:@"isFinished"];
        _isFinished = YES;
        [self didChangeValueForKey:@"isFinished"];
        return;
    }

    if (![NSThread isMainThread])
    {
        [self performSelectorOnMainThread:@selector(start) withObject:nil waitUntilDone:NO];
        return;
    }

    NSLog(@"opeartion for <%@> started.", _url);

    [self willChangeValueForKey:@"isExecuting"];
    _isExecuting = YES;
    [self didChangeValueForKey:@"isExecuting"];

    NSURLRequest * request = [NSURLRequest requestWithURL:_url];
    _connection = [[NSURLConnection alloc] initWithRequest:request
                                                  delegate:self];
    if (_connection == nil)
        [self finish];
}

I might suggest other stylistic improvements to Dave's example, but it's all minor stuff, and I think he got most of the big picture stuff spot on. The failure to check for cancelation was the only obvious big issue that leapt out at me.

Anyway, for discussion of concurrent operations, see the Configuring Operations for Concurrent Execution section of the Concurrency Programming Guide.

Also, when testing huge downloads like these, I'd encourage you to stress test your app with the Network Link Conditioner (available for the Mac/simulator as a download available under "Hardware IO tools" on the "Xcode" - "Open Developer Tool" - "More Developer Tools"; if you enable your iOS device for development, there is also a network link conditioner setting under "General" - "Developer" in the Settings app). A lot of these timeout-related problems don't manifest themselves when we test our apps in our highly optimized scenario of our development environment. It's important to use the network link conditioner to simulate less than ideal, real-world scenarios.