Setting the source IP for a UDP socket

Question

I have a UDP socket that is bound to INADDR_ANY to listen to packets on all the IPs my server has. I\'m sending out replies through the same socket.

Right now the se

Answer 1

Nikolai, using a separate socket and bind(2) for each address or messing with routing tables is often not a feasible option e.g. with dynamic addresses. A single IP_ADDRANY-bound UDP server should be able to appear to respond on the same dynamically-assigned IP address a packet is received on.

Luckily, there is another way. Depending on your system's support you can make use of the IP_PKTINFO socket options to set or receive ancillary data about a message. Ancillary data (via cmsg(3)) is covered in many places online though comp.os.linux.development.system had a full code sample specific to IP_PKTINFO.

The code in the link uses IP_PKTINFO (or IP_RECVDSTADDR depending on the platform) to get the destination address of a UDP message from the ancillary cmsg(3) data. Paraphrased here:

struct msghdr msg;
struct cmsghdr *cmsg;
struct in_addr addr;
// after recvmsg(sd, &msg, flags);
for(cmsg = CMSG_FIRSTHDR(&msg);
    cmsg != NULL;
    cmsg = CMSG_NXTHDR(&msg, cmsg)) {
  if (cmsg->cmsg_level == IPPROTO_IP && cmsg->cmsg_type == IP_PKTINFO) {
    addr = ((struct in_pktinfo*)CMSG_DATA(cmsg))->ipi_addr;
    printf("message received on address %s\n", inet_ntoa(addr));
  }
}

Gene, your question asked how to set the source address on outgoing packets. With IP_PKTINFO it is possible to set the ipi_spec_dst field of the struct in_pktinfo in the ancillary data passed to sendmsg(2). See the post referenced above, cmsg(3), and sendmsg(2) for guidelines on how to create and manipulate the ancillary data in a struct msghdr. An example (no guarantee here) might be:

struct msghdr msg;
struct cmsghdr *cmsg;
struct in_pktinfo *pktinfo;
// after initializing msghdr & control data to CMSG_SPACE(sizeof(struct in_pktinfo))
cmsg = CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level = IPPROTO_IP;
cmsg->cmsg_type = IP_PKTINFO;
cmsg->cmsg_len = CMSG_LEN(sizeof(struct in_pktinfo));
pktinfo = (struct in_pktinfo*) CMSG_DATA(cmsg);
pktinfo->ipi_ifindex = src_interface_index;
pktinfo->ipi_spec_dst = src_addr;
// bytes_sent = sendmsg(sd, &msg, flags);

Note this is different in IPv6: use struct in6_pktinfo::ipi6_addr in both the recvmsg and sendmsg cases.

Note also that Windows does not support an equivalent to ipi_spec_dst in the in_pktinfo struct, so you cannot use this method to set the source address on an outgoing winsock2 packet.

(man pages referenced - getting around 1 hyperlink limit)

http:// linux.die.net/man/2/sendmsg
http:// linux.die.net/man/3/cmsg

Answer 2

I encountered the same problem recently.

What I do to solve this problem is

1. get the interface name from received packet
2. bind socket to specific interface
3. unbind socket

Example:

  struct ifreq ifr;
  ...
  recvmsg(fd, &msg...)
  ...      
  if (msg.msg_controllen >= sizeof(struct cmsghdr))
    for (cmptr = CMSG_FIRSTHDR(&msg); cmptr; cmptr = CMSG_NXTHDR(&msg, cmptr))
      if (cmptr->cmsg_level == SOL_IP && cmptr->cmsg_type == IP_PKTINFO)
      {
        iface_index = ((struct in_pktinfo *)CMSG_DATA(cmptr))->ipi_ifindex;
      }
  if_indextoname(iface_index , ifr.ifr_name);
  mret=setsockopt(fd, SOL_SOCKET, SO_BINDTODEVICE, &ifr, sizeof(ifr));

  sendmsg(...);

  memset(&ifr, 0, sizeof(ifr));
  snprintf(ifr.ifr_name, sizeof(ifr.ifr_name), "");
  mret=setsockopt(fd, SOL_SOCKET, SO_BINDTODEVICE, &ifr, sizeof(ifr));

Answer 3

You either bind(2) to each interface address and manage multiple sockets, or let the kernel do the implicit source IP assignment with INADDR_ANY. There is no other way.

My question would be - why do you need this? Is normal IP routing not working for you?

Answer 4

I thought I'd expand Jeremy's on how to do this for IPv6. Jeremy leaves out a lot of detail, and some documentation (like Linux's man page for ipv6) is just plain wrong. First on some distributions you have to define _GNU_SOURCE, otherwise some of the IPv6 stuff isn't defined:

#define _GNU_SOURCE
#include <netinet/in.h>
#include <sys/types.h>
#include <sys/socket.h>

Next set up the socket in a fairly standard way that listens for all IP packets (ie, both IPv4 and IPv6) on a particular UDP port:

const int on=1, off=0;
int result;
struct sockaddr_in6 sin6;
int soc;

soc = socket(AF_INET6, SOCK_DGRAM, 0);
setsockopt(soc, SOL_SOCKET, SO_REUSEADDR, &on, sizeof(on));
setsockopt(soc, IPPROTO_IP, IP_PKTINFO, &on, sizeof(on));
setsockopt(soc, IPPROTO_IPV6, IPV6_RECVPKTINFO, &on, sizeof(on));
setsockopt(soc, IPPROTO_IPV6, IPV6_V6ONLY, &off, sizeof(off));
memset(&sin6, '\0', sizeof(sin6));
sin6.sin6_family = htons(AF_INET6);
sin6.sin6_port = htons(MY_UDP_PORT);
result = bind(soc, (struct sockaddr*)&sin6, sizeof(sin6));

Notice the code above sets both IP and IPv6 options for an IPv6 socket. Turns out if the packet arrives on an IPv4 address, you will get IP_PKTINFO (ie IPv4) cmsg's even though it is an IPv6 socket, and if you don't enable them they won't be sent. Also notice the IPV6_RECPKTINFO option is set (which isn't mentioned in man 7 ipv6), not IPV6_PKTINFO (which is described wrongly in man 7 ipv6). Now receive a udp packet:

int bytes_received;
struct sockaddr_in6 from;
struct iovec iovec[1];
struct msghdr msg;
char msg_control[1024];
char udp_packet[1500];

iovec[0].iov_base = udp_packet;
iovec[0].iov_len = sizeof(udp_packet);
msg.msg_name = &from;
msg.msg_namelen = sizeof(from);
msg.msg_iov = iovec;
msg.msg_iovlen = sizeof(iovec) / sizeof(*iovec);
msg.msg_control = msg_control;
msg.msg_controllen = sizeof(msg_control);
msg.msg_flags = 0;
bytes_received = recvmsg(soc, &msg, 0);

The next step is to extract the interface and address the UDP packet was received on out of the cmsg:

struct in_pktinfo in_pktinfo;
struct in6_pktinfo in6_pktinfo;
int have_in_pktinfo = 0;
int have_in6_pktinfo = 0;
struct cmsghdr* cmsg;

for (cmsg = CMSG_FIRSTHDR(&msg); cmsg != 0; cmsg = CMSG_NXTHDR(&msg, cmsg))
{
  if (cmsg->cmsg_level == IPPROTO_IP && cmsg->cmsg_type == IP_PKTINFO)
  {
    in_pktinfo = *(struct in_pktinfo*)CMSG_DATA(cmsg);
    have_in_pktinfo = 1;
  }
  if (cmsg->cmsg_level == IPPROTO_IPV6 && cmsg->cmsg_type == IPV6_PKTINFO)
  {
    in6_pktinfo = *(struct in6_pktinfo*)CMSG_DATA(cmsg);
    have_in6_pktinfo = 1;
  }
}

Finally we get to send the response back, using the same destination.

int cmsg_space;

iovec[0].iov_base = udp_response;
iovec[0].iov_len = udp_response_length;
msg.msg_name = &from;
msg.msg_namelen = sizeof(from);
msg.msg_iov = iovec;
msg.msg_iovlen = sizeof(iovec) / sizeof(*iovec);
msg.msg_control = msg_control;
msg.msg_controllen = sizeof(msg_control);
msg.msg_flags = 0;
cmsg_space = 0;
cmsg = CMSG_FIRSTHDR(&msg);
if (have_in6_pktinfo)
{
  cmsg->cmsg_level = IPPROTO_IPV6;
  cmsg->cmsg_type = IPV6_PKTINFO;
  cmsg->cmsg_len = CMSG_LEN(sizeof(in6_pktinfo));
  *(struct in6_pktinfo*)CMSG_DATA(cmsg) = in6_pktinfo;
  cmsg_space += CMSG_SPACE(sizeof(in6_pktinfo));
}
if (have_in_pktinfo)
{
  cmsg->cmsg_level = IPPROTO_IP;
  cmsg->cmsg_type = IP_PKTINFO;
  cmsg->cmsg_len = CMSG_LEN(sizeof(in_pktinfo));
  *(struct in_pktinfo*)CMSG_DATA(cmsg) = in_pktinfo;
  cmsg_space += CMSG_SPACE(sizeof(in_pktinfo));
}
msg.msg_controllen = cmsg_space;
ret = sendmsg(soc, &msg, 0);

Again notice how, if the packet came in via IPv4 we have to put an IPv4 option into the cmsg's even though it is an AF_INET6 socket. At least, that is what you have to do for Linux.

It is a surprising amount of work, but AFAICT it is the minimum you have to do to make a robust UDP server that works in all conceivable Linux environments. Most of it is not required for TCP because it handles multihoming transparently.