What is the best strategy to avoid processing the same item multiple times?

Question

The system I am developing processes items (with a unique ID), at a speed of approximately 1 million per hour. Many of the items it processes have already been processed before,