Efficient data transfer from Java to C++ on windows

Question

I\'m looking to stream lots of data (up to ~1 Gbit) from Java to a C++ application (both on the same machine). I\'m currently using a FIFO on Linux but need a Windows soluti

Answer 1

How about using System.out and System.in?

If that isn't suitable, then Sockets is your best bet.

Answer 2

You should take a look at Protocol Buffers from Google which supports both C++ and Java.

Answer 3

Your fastest solution will be memory mapping a shared segment of memory, and them implementing a ring-buffer or other message passing mechanism. In C++ this is straight forward, and in Java you have the FileChannel.map method which makes it possible.

The next alternative would be to use the stdin/stdout of the two processes. If one can exec the other, this can be quite fast.

Lastly, as you've noted, you can do socket IO. For streaming video this isn't a great option, but if your passing XML around, the overhead will be minimal in comparison to the other processing.

Answer 4

I would use a local socket with negative acknowledgment UDP if the bit rate is going to be too high for TCP (although I would try TCP first and confirm that it is a problem). There should be minimal, if any, packets dropped if you are doing the streaming on the same machine, but the addition of a negative acknowledgement layer will take care of that case for you.

Answer 5

Named pipes would be more efficient than TCP, but how about just using shared memory blocks?

I don't know what primitives exist on the Java side for interfacing with shared memory, but from the C++ side it would be more efficient to access data in shared memory than read it out of either a socket or named pipe. You would have to implement your own flow control and blocking primitives, but these could be fairly straight forward.