How to efficiently remove duplicates from an array without using Set

Question

I was asked to write my own implementation to remove duplicated values in an array. Here is what I have created. But after tests with 1,000,000 elements it took very long ti

Answer 1

public static void main(String[] args) {
        Integer[] intArray = { 1, 1, 1, 2, 4, 2, 3, 5, 3, 6, 7, 3, 4, 5 };
        Integer[] finalArray = removeDuplicates(intArray);
        System.err.println(Arrays.asList(finalArray));
    }

    private static Integer[] removeDuplicates(Integer[] intArray) {
        int count = 0;
        Integer[] interimArray = new Integer[intArray.length];
        for (int i = 0; i < intArray.length; i++) {
            boolean exists = false;
            for (int j = 0; j < interimArray.length; j++) {
                if (interimArray[j]!=null && interimArray[j] == intArray[i]) {
                    exists = true;
                }
            }
            if (!exists) {
                interimArray[count] = intArray[i];
                count++;
            }
        }
        final Integer[] finalArray = new Integer[count];
        System.arraycopy(interimArray, 0, finalArray, 0, count);
        return finalArray;
    }

Answer 2

you can take the help of Set collection

int end = arr.length;
Set<Integer> set = new HashSet<Integer>();

for(int i = 0; i < end; i++){
  set.add(arr[i]);
}

now if you will iterate through this set, it will contain only unique values. Iterating code is like this :

Iterator it = set.iterator();
while(it.hasNext()) {
  System.out.println(it.next());
}

Answer 3

Since you can assume the range is between 0-1000 there is a very simple and efficient solution

//Throws an exception if values are not in the range of 0-1000
public static int[] removeDuplicates(int[] arr) {
    boolean[] set = new boolean[1001]; //values must default to false
    int totalItems = 0;

    for (int i = 0; i < arr.length; ++i) {
        if (!set[arr[i]]) {
            set[arr[i]] = true;
            totalItems++;
        }
    }

    int[] ret = new int[totalItems];
    int c = 0;
    for (int i = 0; i < set.length; ++i) {
        if (set[i]) {
            ret[c++] = i;
        }
    }
    return ret;
}

This runs in linear time O(n). Caveat: the returned array is sorted so if that is illegal then this answer is invalid.

Answer 4

Since this question is still getting a lot of attention, I decided to answer it by copying this answer from Code Review.SE:

You're following the same philosophy as the bubble sort, which is very, very, very slow. Have you tried this?:

• Sort your unordered array with quicksort. Quicksort is much faster than bubble sort (I know, you are not sorting, but the algorithm you follow is almost the same as bubble sort to traverse the array).

• Then start removing duplicates (repeated values will be next to each other). In a for loop you could have two indices: source and destination. (On each loop you copy source to destination unless they are the same, and increment both by 1). Every time you find a duplicate you increment source (and don't perform the copy). @morgano

Answer 5

This is an interview question :Remove duplicates from an array.I shall not use any Set or collections. The complete solution is :

public class Test4 {
public static void main(String[] args) {
     int a[] = {1, 2, 2, 3, 3, 3, 6,6,6,6,6,66,7,65}; 
              int newlength =    lengthofarraywithoutduplicates(a);
              for(int i = 0 ; i < newlength ;i++) {
                  System.out.println(a[i]);
              }//for
}//main

private static int lengthofarraywithoutduplicates(int[] a) {
     int count = 1 ;
     for (int i = 1; i < a.length; i++) {
          int ch = a[i];
          if(ch != a[i-1]) {
              a[count++] = ch;
          }//if
    }//for
    return count;

}//fix

}//end1

Answer 6

You can use an auxiliary array (temp) which in indexes are numbers of main array. So the time complexity will be liner and O(n). As we want to do it without using any library, we define another array (unique) to push non-duplicate elements:

var num = [2,4,9,4,1,2,24,12,4];
let temp = [];
let unique = [];
let j = 0;
for (let i = 0; i < num.length; i++){
  if (temp[num[i]] !== 1){
    temp[num[i]] = 1;
    unique[j++] = num[i];
  }
}
console.log(unique);