How to solve timeout issues caused by bad HTTP persistent connection?

Question

I've been struggling with an HTTP timeout issue recently. After more than one month of investigation I'm quite sure that it is caused by bad HTTP persistent connections. Details are as follows:

1. It is an iOS app.
2. Most users are running iOS 8.
3. I'm using NSURLConnection.
4. iOS 8 has one known keep alive bug but mine is a different issue. More specifically, that bug causes NSURLErrorNetworkConnectionLost but my error is NSURLErrorTimedOut. However, I'm not sure whether my issue is caused by another bug of iOS 8.
5. The behavior of my issue: After some time of using — after some HTTP requests are successfully sent and corresponding responses received — one request would cause NSURLErrorTimedOut, and all following(not too far away from the last one in time to reuse the persistent connection) requests would causes NSURLErrorTimedOut.
6. Some working workaround:
   1. Kill and restart the app.
   2. Turn WiFi connection off on iPhone to force using 3G/4G.
   3. Turn air mode on and then turn it off.
7. My analysis: From the behavior the issue seems to be caused by a gone-bad persistent connection. All the subsequent requests keep using this persistent connection so all fail with NSURLErrorTimedOut. From the workaround we can see all of them work because they cause the bad persistent connection to be dropped and a new persistent connection to be created.

My questions:

1. Have anyone else encountered this issue?
2. Is it a known bug of iOS 8?
3. Is it caused by some unconventional configuration of the servers? I don't control the servers but I know they use nginx 1.6.1 and their engineers are working with me in investigating this issue. What information should I ask them for?
4. Is there any way to force NSURLConnection to not reuse the current persistent connection but to create a new one so I can work around this issue after I detect it in my code?

---

Update:

I successfully mitigated this issue on iOS 8 by using CFNetwork and controlling Connection header directly. However it seems the issue becomes worse on iOS 9.

Since my hope that Apple would fix it on iOS 9 is broken I finally fired a radar: http://www.openradar.me/22770738.

If you also encounter this issue please duplicate my radar, or even better, fire your own radar if you have a more reliably reproducible sample.

Answer 1

After 2 weeks of research, I can give answers to question 3 and 4:

3. nginx's persistent connection timeout is set to 5s on server, which should not be the cause. Server engineers found those timed-out requests are actually normally received and responded. So it is more likely a client side issue. Since I have a minimal reproducible code to rule out my code as the cause, the cause should be in iOS.
4. The only way I found is to use CFNetwork. Higher level API such as NSURLConnection or NSURLSession's Connection header will be overwritten by system.

Answer 2

Same issue here, iOS just try to reuse the connection after server drops it.

Why CFNetwork is NOT Enough

About two years ago, I switched to CFNetwork to workaround this issue, but recently I found it's not possible to implement SSL pinning with CFNetwork API. So now I'm considering go back to NSURLSession.

The Workaround

After some dig around, I found system will NOT reuse connections across NSURLSessions, so creating new sessions within period of time, should solve the issue.

But I also found (at least on macOS): each connection made by NSURLSession can persistence last by 180 seconds, and that connection didn't close by release or reset the session, so you may need to implement some caching mechanism to avoid creating a lots of connections at the same time.

Here is the simple mechanism I'm currently use:

@interface HTTPSession : NSObject

@property (nonatomic, strong) NSURLSession * urlSession;
@property (nonatomic, assign) CFTimeInterval flushTime;

@end

+ (NSURLSession *)reuseOrCreateSession
{
    static NSMutableArray<HTTPSession *> * sessions = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        sessions = [NSMutableArray<HTTPSession *> array];
    });

    const CFTimeInterval serverTimeoutSeconds = 10;
    const CFTimeInterval systemTimeoutSeconds = 40;
    CFTimeInterval now = CFAbsoluteTimeGetCurrent();

    HTTPSession * resultSession = nil;
    for (HTTPSession * session in sessions) {
        CFTimeInterval lifeTime = now - session.flushTime;
        if (lifeTime < serverTimeoutSeconds) {
            resultSession = session;
            break;
        }
        if (lifeTime > systemTimeoutSeconds) {
            resultSession = session;
            resultSession.flushTime = now;
            break;
        }
    }

    if (!resultSession) {
        resultSession = [HTTPSession new];

        NSURLSession * session = [NSURLSession sessionWithConfiguration:[NSURLSessionConfiguration defaultSessionConfiguration]];

        // setup session

        resultSession.urlSession = session;
        resultSession.flushTime = now;

        [sessions addObject:resultSession];
    }

    return resultSession.urlSession;
}

Answer 3

What if you add timestamp to all your request URL? I think this will make each request unique and maybe iOS will establish new connection every time you send request ( I'm not sure. Need trying )