Context We have a homegrown filesystem-backed caching library. We currently have performance problems with one installation due to large number of entries (
Something to look at is how your disk subsystem is arranged. While disks are growing in size rapidly, they are not getting much faster (in access time) Is a different disk arrangement (using more disks) or using SSD drives an option. For example, an SSD has no moving parts and can touch 100K files in 10 seconds. Making the warmup unnecessary.