Efficient way to convert Scala Array to Unique Sorted List

Question

Can anybody optimize following statement in Scala:

// maybe large
val someArray = Array(9, 1, 6, 2, 1, 9, 4, 5, 1, 6, 5, 0, 6) 

// output a sorted list whic         


        

Answer 1

Boxing primitives is going to give you a 10-30x performance penalty. Therefore if you really are performance limited, you're going to want to work off of raw primitive arrays:

def arrayDistinctInts(someArray: Array[Int]) = {    
  java.util.Arrays.sort(someArray)
  var overzero = 0
  var ndiff = 0
  var last = 0
  var i = 0
  while (i < someArray.length) {
    if (someArray(i)<=0) overzero = i+1
    else if (someArray(i)>last) {
      last = someArray(i)
      ndiff += 1
    }
    i += 1
  }
  val result = new Array[Int](ndiff)
  var j = 0
  i = overzero
  last = 0
  while (i < someArray.length) {
    if (someArray(i) > last) {
      result(j) = someArray(i)
      last = someArray(i)
      j += 1
    }
    i += 1
  }
  result
}

You can get slightly better than this if you're careful (and be warned, I typed this off the top of my head; I might have typoed something, but this is the style to use), but if you find the existing version too slow, this should be at least 5x faster and possibly a lot more.

---

Edit (in addition to fixing up the previous code so it actually works):

If you insist on ending with a list, then you can build the list as you go. You could do this recursively, but I don't think in this case it's any clearer than the iterative version, so:

def listDistinctInts(someArray: Array[Int]): List[Int] = {
  if (someArray.length == 0 || someArray(someArray.length-1) <= 0) List[Int]()
  else {
    java.util.Arrays.sort(someArray)
    var last = someArray(someArray.length-1)
    var list = last :: Nil
    var i = someArray.length-2
    while (i >= 0) {
      if (someArray(i) < last) {
        last = someArray(i)
        if (last <= 0) return list;
        list = last :: list
      }
      i -= 1
    }
    list
  }
}

Also, if you may not destroy the original array by sorting, you are by far best off if you duplicate the array and destroy the copy (array copies of primitives are really fast).

And keep in mind that there are special-case solutions that are far faster yet depending on the nature of the data. For example, if you know that you have a long array, but the numbers will be in a small range (e.g. -100 to 100), then you can use a bitset to track which ones you've encountered.