The most efficient way to search for an array of strings in another string

Question

I have a large arrray of strings that looks something like this: String temp[] = new String[200000].

I have another String, let\'s call it bigtext. What I ne

Answer 1

Efficency depends heavily on what is valuable to you.

Are you willing to increase memory for reduced time? Are you willing to increase time for efficent handling of large data sets? Are you willing to increase contention for CPU cores? Are you willing to do pre-processing (perhaps one or more forms of indexing) to reduce the lookup time in a critical section.

With you offering, you indicate the entire portion you want made more efficent, but that means you have excluded any portion of the code or system where the trade-off can be made. This forces one to imagine what you care about and what you don't care about. Odds are excellent that all the posted answers are both correct and incorrect depending on one's point of view.

Answer 2

An alternative approach would be to tokenize the text - let's say split by common punctuation. Then put these tokens in to a Set and then find the intersect with the main container.

Instead of an array, hold the words in a Set too. The intersection can be calculated by simply doing

bidTextSet.retainAll(mainWordsSet);

What remains will be the words that occur in bigText that are in your "dictionary".