Hash function in spark

Question

I\'m trying to add a column to a dataframe, which will contain hash of another column.

I\'ve found this piece of documentation: https://spark.apache

Answer 1

It is Murmur based on the source code.

  /**
   * Calculates the hash code of given columns, and returns the result as an int column.
   *
   * @group misc_funcs
   * @since 2.0.0
   */
  @scala.annotation.varargs
  def hash(cols: Column*): Column = withExpr {
    new Murmur3Hash(cols.map(_.expr))
  }