Find a number where it appears exactly N/2 times

Question

Here is one of my interview question. Given an array of N elements and where an element appears exactly N/2 times and the rest N/2 elements are unique

Answer 1

For worst-case deterministic behavior, O(N) is correct (I've already seen more than one proof in the previous answers).

However, modern algorithmic theory isn't concerned just with worst-case behavior (that's why there are so many other big-somethings besides big-O, even though lazy programmers in a hurry often use big-O even when what they have in mind is closer to big-theta OR big-omega;-), nor just with determinism (withness the Miller-Rabin primality test...;).

Any random sample of K < N items will show no duplicates with a probabllity that < 2**K -- easily and rapidly reduced to essentially as low as you wish no matter what N is (e.g. you could reduce it to less than the probability that a random cosmic ray will accidentally and undetectably flip a bit in your memory;-) -- this observation hardly requires the creativity Rabin and Miller needed to find their probabilistic prime testing approach;-).

This would make a pretty lousy interview question. Similar less-lousy questions are often posed, often mis-answered, and often mis-remembered by unsuccessful candidates. For example, a typical question might be, given an array of N items, not knowing whether there is a majority item, to determine whether there is one, and which one it is, in O(N) time and O(1) auxiliary space (so you can't just set up a hash table or something to count occurrences of different values). "Moore's Voting Approach" is a good solution (probably the best one) to that worthy interviewing question.

Another interesting variation: what if you have 10**18 64-bit numbers (8 Terabytes' worth of data overall, say on a bigtable or clone thereof), and as many machines as you want, each with about 4GB of RAM on a pretty fast LAN, say one that's substantially better than GB ethernet -- how do you shard the problem under those conditions? What if you have to use mapreduce / hadoop? What if you're free to design your own dedicated framework just for this one problem -- could you get better performance than with mapreduce? How much better, at the granularity of back-of-envelope estimation? I know of no published algorithm for THIS variant, so it may be a great test if you want to check general facility of a candidate with highly-distributed approaches to tera-scale computation...