Remove duplicates from a List in C#

Question

Anyone have a quick method for de-duplicating a generic List in C#?

Answer 1

Installing the MoreLINQ package via Nuget, you can easily distinct object list by a property

IEnumerable<Catalogue> distinctCatalogues = catalogues.DistinctBy(c => c.CatalogueCode); 

Answer 2

As a helper method (without Linq):

public static List<T> Distinct<T>(this List<T> list)
{
    return (new HashSet<T>(list)).ToList();
}

Answer 3

Might be easier to simply make sure that duplicates are not added to the list.

if(items.IndexOf(new_item) < 0) 
    items.add(new_item)

Answer 4

An extension method might be a decent way to go... something like this:

public static List<T> Deduplicate<T>(this List<T> listToDeduplicate)
{
    return listToDeduplicate.Distinct().ToList();
}

And then call like this, for example:

List<int> myFilteredList = unfilteredList.Deduplicate();

Answer 5

David J.'s answer is a good method, no need for extra objects, sorting, etc. It can be improved on however:

for (int innerIndex = items.Count - 1; innerIndex > outerIndex ; innerIndex--)

So the outer loop goes top bottom for the entire list, but the inner loop goes bottom "until the outer loop position is reached".

The outer loop makes sure the entire list is processed, the inner loop finds the actual duplicates, those can only happen in the part that the outer loop hasn't processed yet.

Or if you don't want to do bottom up for the inner loop you could have the inner loop start at outerIndex + 1.

Answer 6

Here's a simple solution that doesn't require any hard-to-read LINQ or any prior sorting of the list.

   private static void CheckForDuplicateItems(List<string> items)
    {
        if (items == null ||
            items.Count == 0)
            return;

        for (int outerIndex = 0; outerIndex < items.Count; outerIndex++)
        {
            for (int innerIndex = 0; innerIndex < items.Count; innerIndex++)
            {
                if (innerIndex == outerIndex) continue;
                if (items[outerIndex].Equals(items[innerIndex]))
                {
                    // Duplicate Found
                }
            }
        }
    }