I found a solution to this question, but not sure it's the best way to do this. I store the current languages, clear all the associations, then add back the languages, then save.
languages := user.Languages
DB.Model(&user).Association("Languages").Clear()
user.Languages = languages