Removing Solr duplicate values into multivalued field

Question

My Solr index contains a multivalued field with duplicate values. How can I remove the duplicates ?

Is it possible to overwrite duplicate values into the multivalued fie

Answer 1

You would need to handle it on the Client side to remove the duplicate values.

You can customize the implementation like RemoveDuplicatesTokenFilterFactory (works for same text at same position) to filter out the tokens. Write an extension basically. OR

Also, If using the multivalued field for just faceting ,the value in faceted field is counted just once. So even if you add multiple same values, that would be reflected as a single value in the facet count entry. Have tested this. you too can confirm.

However, the duplicate values would cause the change in the lengthNorm and hence can have an effect on the scoring.