Finding duplicates in a List ignoring a field

Question

I\'ve got a List of Persons and I want to find duplicate entries, consindering all fields except id. So using the equals()-method (and in

Answer 1

Build a Comparator to implement your natural-key ordering and then use a binary-search based deduplication. TreeSet will give you this ability out of the box.

Note that Comparator.compare(a, b) must fulfil the usual antisymmetry, transitivity, consistency and reflexivity requirements or the binary search ordering will fail. You should also make it null-aware (e.g. if the firstname field of one, other or both are null).

A simple natural-key comparator for your Person class is as follows (it is a static member class as you haven't shown if you have accessors for each field).

public class Person {
    public static class NkComparator implements Comparator
    {
        public int compare(Person p1, Person p2)
        {
            if (p1 == null || p2 == null) throw new NullPointerException();
            if (p1 == p2) return 0;
            int i = nullSafeCompareTo(p1.firstname, p2.firstname);
            if (i != 0) return i;
            i = nullSafeCompareTo(p1.lastname, p2.lastname);
            if (i != 0) return i;
            return p1.age - p2.age;
        }
        private static int nullSafeCompareTo(String s1, String s2)
        {
            return (s1 == null)
                    ? (s2 == null) ? 0 : -1
                    : (s2 == null) ? 1 : s1.compareTo(s2);
        }
    }
    private String firstname, lastname;
    private int age;
    private long id;
}

You can then use it to generate a unique list. Use the add method which returns true if and only if the element didn't already exist in the set:

List newList = new ArrayList();
TreeSet nkIndex = new TreeSet(new Person.NkComparator());
for (Person p : originalList)
    if (nkIndex.add(p)) newList.add(p); // to generate a unique list

or swap the final line for this line to output duplicates instead

    if (nkIndex.add(p)) newList.add(p); 

Whatever you do, don't use remove on your original list while you are enumerating it, that's why these methods add your unique elements to a new list.

If you are just interested in a unique list, and want to use as few lines as possible:

TreeSet set = new TreeSet(new Person.NkComparator());
set.addAll(originalList);
List newList = new ArrayList(set);