What is the downside of using smaller set of UDP ports in TURN server?

Question

According to RFC5766, it seems that TURN server should have UDP port 49152 - 65535 open for relay.

In all cases, the server SHOULD only allocate ports fr