Elastic Search - Multiple Field as mapping ID in Spark

Question

I am pretty new to elastic search. I am using elasticsearch-hadoop 6.2.4 version and I am reading the files from HDFS, converting to bean object and wr

Answer 1

Or you can use sha2 function by generating hashed id after concatenating multiple columns.

Answer 2

According to Elastic Documentation; mapping id option is take 1 column name, so; you can not set multiple columns as id. but you can solve this issue by creating a new column with this value like this:

dataSet.withColumn('id', CustomerID + OrderID)

Answer 3

No, You can't set multiple attributes as "es.mapping.id" . One Thing you can do is, what ever composite Id you want, Create it and append it to the Dataframe and use the same.