Unable to sustain a Constant Speed while Uploading files in the background with NSURLSession

Question

I am trying to upload some 100 images to S3 in the background with AFURLSessionManager in small batches of 10 like what is being done here- Manage the number of active tasks in a background NSURLSession

I am using a shared NSURLSession and adding tasks according more tasks to this when some tasks are completed. Average size of each file is about 1.6 MB and the number of tasks that are guaranteed to run per a task queue is 5

Here is my method for adding the tasks: (also available as an easier-to-read gist)

    - (void) addTasksToSessionWithTaskObject:(Task*)taskObject withSessionInitialisationNeeded:(BOOL) needed{

        NSString *filePath = [[NSBundle mainBundle] pathForResource:pathForResourceFile ofType:resourceFileType];
        S3PutObjectRequest *putObjectRequest = [[S3PutObjectRequest alloc] initWithKey:targetFileKey
                                                                              inBucket:_bucketname];
        putObjectRequest.cannedACL = [S3CannedACL publicReadWrite];
        putObjectRequest.filename = filePath;
        putObjectRequest.contentType = [resourceFileType isEqualToString:@"MOV"] ? @"movie/mov" : @"image/jpg";
        putObjectRequest.endpoint = @"http://s3.amazonaws.com";
        putObjectRequest.contentLength=[[[NSFileManager defaultManager]
                                         attributesOfItemAtPath:filePath error:nil] fileSize];
        putObjectRequest.delegate = self;
        [putObjectRequest configureURLRequest];
        NSMutableURLRequest *request = [s3Client signS3Request:putObjectRequest];
        NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc]initWithURL:[NSURL URLWithString:[NSString stringWithFormat:@"http://s3.amazonaws.com/UploadTest/%@",taskObject.fileKey]]];

        [request2 setHTTPMethod:request.HTTPMethod];
        [request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];

        if(needed) {

                sharedSession = [self backgroundSession];   
        }
   NSURLSessionUploadTask *task = [sharedSession uploadTaskWithRequest:request2           fromFile:forFileUrl];

    task.taskDescription = pathForResourceFile;
    [currentlyActiveTaskIdArray addObject:@([task taskIdentifier])];

    [task resume];
}

And this what I've done with the delegate

- (void)URLSession:(NSURLSession *)sessionI task:(NSURLSessionTask *)task
didCompleteWithError:(NSError *)error{

    dispatch_async(dispatch_get_main_queue(), ^{

        __block UIBackgroundTaskIdentifier bgTaskI = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
            [[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
        }];

        if([currentlyActiveTaskIdArray containsObject:@([task taskIdentifier])]){
            [currentlyActiveTaskIdArray removeObject:@([task taskIdentifier])];
        }
        if(currentlyActiveTaskIdArray.count < LOWER_SLAB_FOR_TASKS + 1){   
            [self initiateS3UploadForSetOfTasksIsItBeginningOfUpload:NO];
        }
        [[UIApplication sharedApplication] endBackgroundTask:bgTaskI];
    }); 
}

Here is the Code to add more tasks

 - (void) initiateS3UploadForSetOfTasksIsItBeginningOfUpload:(BOOL)beginning{
        int i=0;
        for(Task *eachTaskObject in tasksArray){
            if(i < numberOfTasksTobeAdded){  
                [self addTasksToSessionWithTaskObject:eachTaskObject WithSessionInitialisationNeeded:NO];
                i++;
            }
        }
    }

I've been running tests with 100 files in Foreground mode and Background mode. In Foreground mode, it uploads the files at a consistant, steady and constant speed, it completes 90 files in the first 3 minutes, and the remaining 10 files in 20 seconds.

When I run the app in Background mode, I would expect it to upload the first 90 files just as fast as it did in the 3 minute Foreground window, and then slow down after that... but that's not the case. In Background mode, it uploads 15 files in the first minute, then it starts slowing down... a lot. It starts uploading in 8 file batches in slower and slower intervals: 1 minute, 3 minutes, 5 minutes, 10 minutes, and now 17 minutes. We're at 65 files 46 minutes in.

Is there a way to either keep it fast for at least the fist 3 minutes, or keep consistent speed in the background?

UPDATE: Following the comments from Clay here Ive switched back to NSURLSession from AFURLSessionManager because as he points out using block based callbacks is an extremely risky business with NSURLSession. Further I've played with HTTPMaximumConnectionsPerHost and set this around 10-this has given better results but nowhere near what I would want to be.

Answer 1

From what I can tell, setTaskDidCompleteBlock: is not an Apple API, NSURLSession-associated method. It is an AFURLSessionManager method (docs). If you are using AFNetworking on this, then you need to be announcing that bold, top, front and center. That is not the same, at all, as using NSURLSession. I would guess AFNetworking's background NSURLSession-based implementation comes with its own foibles and idiosyncrasies.

For my part, whatever success I've had with sustained background NSURLSession uploads are using only the stock API.

---

Addressing questions, etc.

• Regarding AFNetworking: we use it for general web api I/O. At the time NSURLSession came out, AFNetworking really didn't robustly support app-in-background ops, so I didn't use it. Perhaps because I went through the background NSURLSession pain & hazing, I look askance at AFNetworking backgrounding under the rubric of "Now you have two problems". But maybe they have cracked the nut by now.
  

  • I strive for one NSURLSession. I started out being cavalier about creation & destruction of sessions, but found this made for some truly gnarly problems. Experiences seem to vary on this.

  • I use the default HTTPMaximumConnectionsPerHost, no problems there. The Apple docs are silent on the default value, but here's what lldb tells me in the random particular device/OS I chose:
    (lldb) p [config HTTPMaximumConnectionsPerHost] (NSInteger) $0 = 4
    If you are having troubles with backgrounding slowing down, I doubt tweaking this is on the right track.

  • FWIW, background NSURLSessions do not support the block interfaces, delegate only.