How to sync CoreData and a REST web service asynchronously and the same time properly propagate any REST errors into the UI

Question

Hey, I\'m working on the model layer for our app here.

Some of the requirements are like this:

1. It should work on iPhone OS 3.0+.
2. The source of

Answer 1

This becomes a sync problem and not one easy to solve. Here's what I'd do: In your iPhone UI use one context and then using another context (and another thread) download the data from your web service. Once it's all there go through the sync/importing processes recommended below and then refresh your UI after everything has imported properly. If things go bad while accessing the network, just roll back the changes in the non UI context. It's a bunch of work, but I think it's the best way to approach it.

Core Data: Efficiently Importing Data

Core Data: Change Management

Core Data: Multi-Threading with Core Data

Answer 2

You should really take a look at RestKit (http://restkit.org) for this use case. It is designed to solve the problems of modeling and syncing remote JSON resources to a local Core Data backed cache. It supports an offline mode for working entirely from the cache when there is no network available. All syncing occurs on a background thread (network access, payload parsing, and managed object context merging) and there is a rich set of delegate methods so you can tell what is going on.

Answer 3

You need a callback function that's going to run on the other thread (the one where actual server interaction happens) and then put the result code/error info a semi-global data which will be periodically checked by UI thread. Make sure that the wirting of the number that serves as the flag is atomic or you are going to have a race condition - say if your error response is 32 bytes you need an int (whihc should have atomic acces) and then you keep that int in the off/false/not-ready state till your larger data block has been written and only then write "true" to flip the switch so to speak.

For the correlated saving on the client side you have to either just keep that data and not save it till you get OK from the server of make sure that you have a kinnf of rollback option - say a way to delete is server failed.

Beware that it's never going to be 100% safe unless you do full 2-phase commit procedure (client save or delete can fail after the signal from the server server) but that's going to cost you 2 trips to the server at the very least (might cost you 4 if your sole rollback option is delete).

Ideally, you'd do the whole blocking version of the operation on a separate thread but you'd need 4.0 for that.

Answer 4

There are three basic components:

1. The UI Action and persisting the change to CoreData
2. Persisting that change up to the server
3. Refreshing the UI with the response of the server

An NSOperation + NSOperationQueue will help keep the network requests orderly. A delegate protocol will help your UI classes understand what state the network requests are in, something like:

@protocol NetworkOperationDelegate
  - (void)operation:(NSOperation *)op willSendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
  - (void)operation:(NSOperation *)op didSuccessfullySendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
  - (void)operation:(NSOperation *)op encounteredAnError:(NSError *)error afterSendingRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity;
@end

The protocol format will of course depend on your specific use case but essentially what you're creating is a mechanism by which changes can be "pushed" up to your server.

Next there's the UI loop to consider, to keep your code clean it would be nice to call save: and have the changes automatically pushed up to the server. You can use NSManagedObjectContextDidSave notifications for this.

- (void)managedObjectContextDidSave:(NSNotification *)saveNotification {
  NSArray *inserted = [[saveNotification userInfo] valueForKey:NSInsertedObjects];
  for (NSManagedObject *obj in inserted) {
    //create a new NSOperation for this entity which will invoke the appropraite rest api
    //add to operation queue
  }

  //do the same thing for deleted and updated objects
}

The computational overhead for inserting the network operations should be rather low, however if it creates a noticeable lag on the UI you could simply grab the entity ids out of the save notification and create the operations on a background thread.

If your REST API supports batching, you could even send the entire array across at once and then notify you UI that multiple entities were synchronized.

The only issue I foresee, and for which there is no "real" solution is that the user will not want to wait for their changes to be pushed to the server to be allowed to make more changes. The only good paradigm I have come across is that you allow the user to keep editing objects, and batch their edits together when appropriate, i.e. you do not push on every save notification.