Data structure for matching sets

Question

I have an application where I have a number of sets. A set might be
{4, 7, 12, 18}
unique numbers and all less than 50.

I then have several data items:
1 {1,

Answer 1

Another idea is to completely prehunt your elephants.

Setup

Create a 64 bit X 50,000 element bit array.

Analyze your search set, and set the corresponding bits in each row.

Save the bit map to disk, so it can be reloaded as needed.

Searching

Load the element bit array into memory.

Create a bit map array, 1 X 50000. Set all of the values to 1. This is the search bit array

Take your needle, and walk though each value. Use it as a subscript into the element bit array. Take the corresponding bit array, then AND it into the search array.

Do that for all values in your needle, and your search bit array, will hold a 1, for every matching element.

Reconstruct

Walk through the search bit array, and for each 1, you can use the element bit array, to reconstruct the original values.