问题
I am working on a data mining algorithm where I need to tokenize the string using multiple words. I have a separate file which contain all the stopwords. What I need to do is to tokenize the input string by any of the word (stopword) working as delimiter.
For eg.
If the file contains stopwords as
a
is
and
of
that
and the input string comes to be
"a computer cluster consists of a set of loosely connected computers that work together"
the output comes to be
computer cluster consists
set
loosely connected computers
work together
Checking string against all the stopwords recursively would be very time consuming? Is there any good method for this?
回答1:
Construct a regular expression of the form
delim1|delim2|delim3
then use String's split() method to split the text by any of the delimiters.
In order to construct the regexp, read each delimiter, and pass it to Pattern.quote before appending to the regex that you build. This would let your delimiters use regex metacharacters as well.
来源：https://stackoverflow.com/questions/13223183/string-split-using-multiple-delimiters-in-java