How to map a column to create a new column in spark sql dataframe?

Question

In python and pandas, I can create a new column like this: // using two columns in pandas dataframe to create a dict. dict1 = dict(zip(data["id"], data["duration&