From what I know, all the MAC addresses that the packet gone through are in the packet. This is because that each packet that goes in a certain path, should also be returned
Before understanding that why the server cannot access the MAC address of its client, lets understand how a data packet traverse in a network.
Case 1 :
The destination IP address will be known to the network application or obtained from a DNS server in case the application knows a domain name only. Then the host will compare the destination IP address with its own IP address and subnet mask. If the destination host is within the source host's network, the source host will send an ARP request onto the wire asking "who has that IP address?" and the destination host will reply with its MAC address.
Case 2 : (which is generally the case over the Internet)
If the destination host is not within the source host's network, lets say when we are browsing Google.com, the DNS will resolve the Name to IP. the source host will send an ARP request asking "who has the IP address of my default gateway?". In this case since it is out of our network, we will send ARP request to our Default Gateway(A default gateway is the node in a computer network using the internet protocol suite that serves as the forwarding host (router) to other networks), and DG replies with its MAC. You send the packet with Destination IP of Google and Destination MAC of our DG. When the DG recieves this Frame with its own destination MAC and some other destination IP address in the IP Header, it knows that the packet is not destined to itself, but only traversing. Then the ip lookup for google, followed by ARP for next hop etc continues..
So you see, the data link layer that does physical addressing never knew the MAC address of the destination, it only knew the MAC address of next HOP(router). The MAC address of the next router is obtained through ARP request. So how does the receiver know MAC address of its sender. Well it never knew nor it can because every time the packet was received by a router, it added its own MAC address in the source MAC address field or rather say it replaced it with its own. Hence the server will never be able to know MAC address of its receiver as it never was there in the packet. Its just the IP address that remains.