Remove substrings inside a list with better than O(n^2) complexity

Question

I have a list with many words (100.000+), and what I\'d like to do is remove all the substrings of every word in the list.

So for simplicity, let\'s imagine that I have

Answer 1

@wim is correct.

Given an alphabet of fixed length, the following algorithm is linear in the overall length of text. If the alphabet is of unbounded size, then it will be O(n log(n)) instead. Either way it is better than O(n^2).

Create an empty suffix tree T.
Create an empty list filtered_words
For word in words:
    if word not in T:
        Build suffix tree S for word (using Ukkonen's algorithm)
        Merge S into T
        append word to filtered_words