Lucene Analyzer chain: ShingleFilter without filler tokens
问题 In my analyzer chain, ShingleFilter comes after stopword filter. As mentioned in the docs, ShingleFilter handles position increments > 1 by inserting filler tokens (tokens with termtext "_"). For example : "please divide this sentence into biword shingles" Shingles of size 2 : please divide, divide _, _ sentence, sentence _, _ biword, biword shingles (assuming that "this, "into" are stopwords) I would like to eliminate those shingles with the filler tokens, i.e. my desired output contains