socketchannel.write() becomes very slow when message size is large

Question

In my program using java nio, the socketchannel.write() becomes very slow when it tries to write 10 KB messages consecutively. The measured time for writing a complete 10 KB message is between 160 ms and 200 ms. But the time for writing a complete 5 KB message is only takes 0.8 ms.

In the selector, I only have Selection.OP_READ and do not handle Selection.OP_WRITE. When a large complete message is received, it is written to another receiver 4 times.

Is anyone accounter same problem? There is a post about socketchannel.write() slow. My question is how to alternate change between OP_READ and OP_WRITE?

If I add an inerval e.g, 150 ms, the response time is reduced. Is there any way to find when the buffer is full so I can let the program waits. My operating system is windows xp.

Thanks.

I follow EPJ suggestion by checking the number of written bytes. But the response time is still high. I post part of my code here and would like to examine whether there is wrong with my code.

// this is the writeData() part using nio:

      while (buffer.hasRemaining()) {   
        try {           
                buffer.flip();                  
                n = socket.write(buffer);           
                if(n == 0) {                
                    key.interestOps(SelectionKey.OP_WRITE);
                    key.attach(buffer);             
                    break;
                }                               
        } catch (IOException e) {               
            e.printStackTrace();
        } finally {
            buffer.compact();
        }
    }   

    if(buffer.position()==0) {                  
        key.interestOps(SelectionKey.OP_READ);
    }

Answer 1

If write takes more than 20 micro-seconds, I would suggest you have a buffer full issue. I assume you are using blocking NIO. When the send buffer is not full it usually takes between 5 - 20 micro-seconds. In the past I have configured my server to kill any slow consumer which takes 2 ms to write. (Possibly a bit aggressive. ;)

You could try increasing the size of the send buffer (Socket.setSendBufferSize(int), which is also available for SocketChannels), but it would appear you are trying to send more data than your bandwidth allows.

10 KB is not a large message, the typical send buffer size is 64 KB, so for it to be full you would need to have 6-7 messages unsent. This might explain way 5KB is relatively fast.

Answer 2

I suggest that your reading process is slow and that this is causing its receive buffer to back up, which is causing your send buffer to back up, which stalls your sends.

Or else you haven't written the code correctly for non-blocking mode. If you get a zero result from the write() method, you must (a) change the interestOps to OP_WRITE and (b) return to your select loop. When you get OP_WRITE you must then repeat the write; if you wrote all the data, change the interestOps back to OP_READ, otherwise leave everything as is and wait for the next OP_WRITE. If you attempt to loop while writing in non-blocking mode even in the presence of zero-length writes you will just spin, wasting CPU cycles and time.

Modulo bugs:

while (buffer.position() > 0)
{
  try
  {
    buffer.flip();
    int count = ch.write(buffer);
    if (count == 0)
    {
      key.interestOps(SelectionKey.OP_WRITE);
      break;
    }
  }
  finally
  {
    buffer.compact();
  }
}
if (buffer.position() == 0)
{
  key.interestOps(SelectionKey.OP_READ);
}