I am trying to create a new column based on both columns. Say I want to create a new column z, and it should be the value of y when it is not missing and be the value of x when
You can use apply with option axis=1. Then your solution is pretty concise.
apply
axis=1
df[z] = df.apply(lambda row: row.y if pd.notnull(row.y) else row.x, axis=1)