How would one transfer files larger than 2,147,483,646 bytes (~2 GiB) with Win32 TransmitFile()?

问题

Quoted from MSDN entry for TransmitFile:

The maximum number of bytes that can be transmitted using a single call to the TransmitFile function is 2,147,483,646, the maximum value for a 32-bit integer minus 1. The maximum number of bytes to send in a single call includes any data sent before or after the file data pointed to by the lpTransmitBuffers parameter plus the value specified in the nNumberOfBytesToWrite parameter for the length of file data to send. If an application needs to transmit a file larger than 2,147,483,646 bytes, then multiple calls to the TransmitFile function can be used with each call transferring no more than 2,147,483,646 bytes. Setting the nNumberOfBytesToWrite parameter to zero for a file larger than 2,147,483,646 bytes will also fail since in this case the TransmitFile function will use the size of the file as the value for the number of bytes to transmit.

Alright. Sending a file of size 2*2,147,483,646 bytes (~ 4 GiB) with TransmitFile would then have to be divided into two parts at minimum (e.g. 2 GiB + 2 GiB in two calls to TransmitFile). But how exactly would one go about doing that, while preferably also keeping the underlying TCP connection alive in between?

When the file is indeed <=2,147,483,646 bytes in size, one could just write:

HANDLE fh = CreateFile(filename, GENERIC_READ, FILE_SHARE_READ, NULL, 
 OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL | FILE_FLAG_SEQUENTIAL_SCAN, NULL);   

TransmitFile(SOCK_STREAM_socket, fh, 0, 0, NULL, NULL, TF_DISCONNECT);

to let Windows handle all the lower-level stuff (caching, chunking the data up into pieces for efficient transmission etc. However, unlike the comparable Linux sendfile() syscall, there is no immediately obvious offset argument in the call (although the fifth argument, LPOVERLAPPED lpOverlapped probably is exactly what I'm looking for). I suppose I could hack something together, but I'm also looking for a graceful, good practice Win32 solution from someone who actually knows about this stuff.

You can use the lpOverlapped parameter to specify a 64-bit offset within the file at which to start the file data transfer by setting the Offset and OffsetHigh member of the OVERLAPPED structure. If lpOverlapped is a NULL pointer, the transmission of data always starts at the current byte offset in the file.

So, for lack of a minimal example readily available on the net, which calls are necessary to accomplish such a task?

回答1:

Managed to figure it out based on the comments.

So, if LPOVERLAPPED lpOverlapped is a null pointer, the call starts transmission at the current file offset of the file (much like the Linux sendfile() syscall and its off_t *offset parameter). This offset (pointer) can be manipulated with SetFilePointerEx, so one could write:

#define TRANSMITFILE_MAX ((2<<30) - 1)

LARGE_INTEGER total_bytes;
memset(&total_bytes, 0, sizeof(total_bytes));

while (total_bytes < filesize) {
     DWORD bytes = MIN(filesize-total_bytes, TRANSMITFILE_MAX);
     if (!TransmitFile(SOCK_STREAM_socket, fh, bytes, 0, NULL, NULL, 0))
     { /* error handling */ }

     total_bytes.HighPart += bytes;
     SetFilePointerEx(fh, total_bytes, NULL, FILE_BEGIN);
}

closesocket(SOCK_STREAM_socket);

to accomplish the task. Not very elegant imo, but it works.

来源：https://stackoverflow.com/questions/18240501/how-would-one-transfer-files-larger-than-2-147-483-646-bytes-2-gib-with-win32

标签

sockets

winapi

visual-c++

transmitfile