In my case what actually helped without increasing the sort buffer size was to create indexes for the fields that I was ordering on. I'm not sure if it would work when joining tables but it will work when trying to order a single table which is very large.