Data structure for matching sets

Question

I have an application where I have a number of sets. A set might be
{4, 7, 12, 18}
unique numbers and all less than 50.

I then have several data items:
1 {1,

Answer 1

A possible way to divvy up the list of bitmaps, would be to create an array of (Compiled Nibble Indicators)

Let's say one of your 64 bit bitmaps has the bit 0 to bit 8 set.
In hex we can look at it as 0x000000000000001F

Now, let's transform that into a simpler and smaller representation. Each 4 bit Nibble, either has at least one bit set, or not. If it does, we represent it as a 1, if not we represent it as a 0.

So the hex value reduces to bit pattern 0000000000000011, as the right hand 2 nibbles have are the only ones that have bits in them. Create an array, that holds 65536 values, and use them as a head of linked lists, or set of large arrays....

Compile each of your bit maps, into it's compact CNI. Add it to the correct list, until all of the lists have been compiled.

Then take your needle. Compile it into its CNI form. Use that to value, to subscript to the head of the list. All bitmaps in that list have a possibility of being a match. All bitmaps in the other lists can not match.

That is a way to divvy them up.

Now in practice, I doubt a linked list would meet your performance requirements.

If you write a function to compile a bit map to CNI, you could use it as a basis to sort your array by the CNI. Then have your array of 65536 heads, simply subscript into the original array as the start of a range.

Another technique would be to just compile a part of the 64 bit bitmap, so you have fewer heads. Analysis of your patterns should give you an idea of what nibbles are most effective in partitioning them up.

Good luck to you, and please let us know what you finally end up doing.

Evil.