Core Data memory usage while importing large dataset

Question

I\'m now stuck for about two weeks with a nasty Core Data problem. I read lots of blogpost, articles and SO questions/answers but I\'m still not able to solve my problem.

<

Answer 1

Robin,

I have a similar problem which I solved differently than you have. In your case, you have a third, IMO, redundant MOC, the parent MOC. In my case, I let the two MOCs communicate, in an old school fashion, through the persistent store coordinator via the DidSave notifications. The new block oriented APIs make this much simpler and robust. This lets me reset the child MOCs. While you gain a performance advantage from your third MOC, it isn't that great of an advantage over the SQLite row cache which I exploit. Your path consumes more memory. Finally, I can, by tracking the DidSave notifications, trim items as they are created.

BTW, you are also probably suffering from a massive increase in the size of your MALLOC_TINY and MALLOC_SMALL VM regions. My trailing trimming algorithm lets the allocators reuse space sooner and, hence, retards the growth of these problematic regions. These regions are, in my experience, due to their large resident memory footprint a major cause for my app, Retweever, being killed. I suspect your app suffers the same fate.

When the memory warnings come, I call the below snippet:

[self.backgroundMOC performBlock: ^{ [self.backgroundMOC reset]; }];

[self.moc save];

[self.moc.registeredObjects trimObjects];

-[NSArray(DDGArray) trimObjects] just goes through an array and refreshes the object, thus trimming them.

In summary, Core Data appears to implement a copy on write algorithm for items that appear in many MOCs. Hence, you have things retained in unexpected ways. I focus upon breaking these connections after import to minimize my memory footprint. My system, due to the SQLite row cache, appears to performa acceptably well.

Andrew