Isn't your answer in the LocalEndPoint variable, which is the EndPoint of the packet's source, i.e., the dude on the other end. Note that I would probably rename this variable something like "remoteEP", and initialize it to something non-specific to avoid confusion.