This document says std::list
is inefficient:
std::list is an extremely inefficient class that is rarely useful. It performs a heap allocation
I would suggest doing exactly what @Yves Daoust says, except instead of using a linked list for the free list, use a vector. Push and pop the free indices on the back of the vector. This is amortized O(1) insert, lookup, and delete, and doesn't involve any pointer chasing. It also doesn't require any annoying allocator business.