MongoDB very slow deletes

Question

I\'ve got a small replica set of three mongod servers (16GB RAM each, at least 4 CPU cores and real HDDs) and one dedicated arbiter. The replicated data has about 100,000,0

Answer 1

This is happening because even though

db.repo.remove({"date" : {"$lt" : new Date(1362096000000)}})

looks like a single command it's actually operating on many documents - as many as satisfy this query.

When you use replication, every change operation has to be written to a special collection in the local database called oplog.rs - oplog for short.

The oplog has to have an entry for each deleted document and every one of those entries needs to be applied to the oplog on each secondary before it can also delete the same record.

One thing I can suggest that you consider is TTL indexes - they will "automatically" delete documents based on expiration date/value you set - this way you won't have one massive delete and instead will be able to spread the load more over time.