When you try to sort by distance in ES, the result usually contains the distance between matches and your search location:
"hits": { "total&q