How do I use NSOperationQueue with NSURLSession?

I'm trying to build a bulk image downloader, where images can be added to a queue on the fly to be downloaded, and I can find out the progress and when they're done downloading.

Through my reading it seems like NSOperationQueue for the queue functionality and NSURLSession for the network functionality seems like my best bet, but I'm confused as to how to use the two in tandem.

I know I add instances of NSOperation to the NSOperationQueue and they get queued. And it seems I create a download task with NSURLSessionDownloadTask, and multiple if I need multiple tasks, but I'm not sure how I put the two together.

NSURLSessionDownloadTaskDelegate seems to have all the information I need for download progress and completion notifications, but I also need to be able to stop a specific download, stop all the downloads, and deal with the data I get back from the download.

Your intuition here is correct. If issuing many requests, having an NSOperationQueue with maxConcurrentOperationCount of 4 or 5 can be very useful. In the absence of that, if you issue many requests (say, 50 large images), you can suffer timeout problems when working on a slow network connection (e.g. some cellular connections). Operation queues have other advantages, too (e.g. dependencies, assigning priorities, etc.), but controlling the degree of concurrency is the key benefit, IMHO.

If you are using completionHandler based requests, implementing operation-based solution is pretty trivial (it's the typical concurrent NSOperation subclass implementation; see the Configuring Operations for Concurrent Execution section of the Operation Queues chapter of the Concurrency Programming Guide for more information).

If you are using the delegate based implementation, things start to get pretty hairy pretty quickly, though. This is because of an understandable (but incredibly annoying) feature of NSURLSession whereby the task-level delegates are implemented at the session-level. (Think about that: Two different requests that require different handling are calling the same delegate method on the shared session object. Egad!)

Wrapping a delegate-based NSURLSessionTask in an operation can be done (I, and others I'm sure, have done it), but it involves an unwieldy process of having the session object maintain a dictionary cross referencing task identifiers with task operation objects, have it pass these task delegate methods passed to the task object, and then have the task objects conform to the various NSURLSessionTask delegate protocols. It's a pretty significant amount of work required because NSURLSession doesn't provide a maxConcurrentOperationCount-style feature on the session (to say nothing of other NSOperationQueue goodness, like dependencies, completion blocks, etc.).

And it's worth pointing out that operation-based implementation is a bit of a non-starter with background sessions, though. Your upload/download tasks will continue to operate well after the app has been terminated (which is a good thing, that's fairly essential behavior in a background request), but when your app is restarted, the operation queue and all of its operations are gone. So you have to use a pure delegate-based NSURLSession implementation for background sessions.

Conceptually, NSURLSession is an operation queue. If you resume an NSURLSession task and breakpoint on the completion handler, the stack trace can be quite revealing.

Here's an excerpt from the ever faithful Ray Wenderlich's tutorial on NSURLSession with an added NSLog statement to breakpoint on executing the completion handler:

NSURLSession *session = [NSURLSession sharedSession];
[[session dataTaskWithURL:[NSURL URLWithString:londonWeatherUrl]
          completionHandler:^(NSData *data,
                              NSURLResponse *response,
                              NSError *error) {
            // handle response
            NSLog(@"Handle response"); // <-- breakpoint here       

  }] resume];

Above, we can see the completion handler being executed in Thread 5 Queue: NSOperationQueue Serial Queue.

So, my guess is that each NSURLSession maintains it's own operation queue, and each task added to a session is - under the hood - executed as an NSOperation. Therefore, it doesn't make sense to maintain an operation queue that controls NSURLSession objects or NSURLSession tasks.

NSURLSessionTask itself already offers equivalent methods such as cancel, resume, suspend, and so on.

It's true that there is less control than you would have with your own NSOperationQueue. But then again, NSURLSession is a new class the purpose of which is undoubtably to relieve you of that burden.

Bottom line: if you want less hassle - but less control - and trust Apple to perform the network tasks competently on your behalf, use NSURLSession. Otherwise, roll your own with NSURLConnection and your own operation queues.

Update: The executing and finishing properties hold the knowledge about the status of the current NSOperation. Once you finishing is set to YES and executing to NO, your operation is considered as finished. The correct way of dealing with it does not require a dispatch_group and can simply be written as an asynchronous NSOperation:

  - (BOOL) isAsynchronous {
     return YES;
  }

  - (void) main
    {
       // We are starting everything
       self.executing = YES;
       self.finished = NO;

       NSURLSession * session = [NSURLSession sharedInstance];

       NSURL *url = [NSURL URLWithString:@"http://someurl"];

       NSURLSessionDataTask * dataTask = [session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error){

          /* Do your stuff here */

         NSLog("Will show in second");

         self.executing = NO;
         self.finished = YES;
       }];

       [dataTask resume]
   }

The term asynchronous is quite misleading and does not refers to the differentiation between UI (main) thread and background thread.

If isAsynchronous is set to YES, it means that some part of the code is executed asynchronously regarding the main method. Said differently: an asynchronous call is made inside the main method and the method will finish after the main method finishes.

I have some slides about how to handle concurrency on apple os: https://speakerdeck.com/yageek/concurrency-on-darwin.

Old answer: You could try the dispatch_group_t. You can think them as retain counter for GCD.

Imagine the code below in the main method of your NSOperation subclass :

- (void) main
{

   self.executing = YES;
   self.finished = NO;

   // Create a group -> value = 0
   dispatch_group_t group = dispatch_group_create();

   NSURLSession * session = [NSURLSession sharedInstance];

   NSURL *url = [NSURL URLWithString:@"http://someurl"];

    // Enter the group manually -> Value = Value + 1
   dispatch_group_enter(group); ¨

   NSURLSessionDataTask * dataTask = [session dataTaskWithURL:url completionHandler:^(NSData *data, NSURLResponse *response, NSError *error){


      /* Do your stuff here */

      NSLog("Will show in first");

      //Leave the group manually -> Value = Value - 1
      dispatch_group_leave(group);
   }];

   [dataTask resume];

  // Wait for the group's value to equals 0
  dispatch_group_wait(group, DISPATCH_TIME_FOREVER);

  NSLog("Will show in second");

  self.executing = NO;
  self.finished = YES;
}

With NSURLSession you don't manually add any operations to a queue. You use the method - (NSURLSessionDataTask *)dataTaskWithRequest:(NSURLRequest *)request on NSURLSession to generate a data task which you then start (by calling the resume method).

You are allowed to provide the operation queue so you can control the properties of the queue and also use it for other operations if you wanted.

Any of the usual actions you would want to take on a NSOperation (i.e. start, pause, stop, resume) you perform on the data task.

To queue up 50 images to download you can simply create 50 data tasks which the NSURLSession will properly queue up.

If you're using OperationQueue and don't want each operation to create many simultaneous network requests, you can simply call queue.waitUntilAllOperationsAreFinished() after each operation is added to the queue. They will now only execute after the previous one is completed, significantly reducing the amount of simultaneous network connections.