问题
I'm looking to stream lots of data (up to ~1 Gbit) from Java to a C++ application (both on the same machine). I'm currently using a FIFO on Linux but need a Windows solution too.
The most cross-platform method seems to be a local socket, but: a) won't I get huge overhead from TCP checksumming and copying to & from kernel space, and b) won't the average user's firewall try to inspect the and maybe block the connection?
It seems like a safer solution may be to use JNI and the Named Pipe API (\.\pipe\blah), making a god-awful platform-specific mess of both sides of the connection.
Are these really my 2 best options (and which would people recommend?) Thanks!
回答1:
You should take a look at Protocol Buffers from Google which supports both C++ and Java.
回答2:
Named pipes would be more efficient than TCP, but how about just using shared memory blocks?
I don't know what primitives exist on the Java side for interfacing with shared memory, but from the C++ side it would be more efficient to access data in shared memory than read it out of either a socket or named pipe. You would have to implement your own flow control and blocking primitives, but these could be fairly straight forward.
回答3:
I would use a local socket, which is, as you state, the most cross-platform method.
Kernel-User space copies should not be an issue since any other method you could choose would requiere this kind of copy except for maybe shared memory. It is available on every Unix system and also Windows has its way of doing it
To use shared memory in Java the only way is to implement it by means of your own .DLL/.SO and JNI to access it.
回答4:
Your fastest solution will be memory mapping a shared segment of memory, and them implementing a ring-buffer or other message passing mechanism. In C++ this is straight forward, and in Java you have the FileChannel.map method which makes it possible.
The next alternative would be to use the stdin/stdout of the two processes. If one can exec the other, this can be quite fast.
Lastly, as you've noted, you can do socket IO. For streaming video this isn't a great option, but if your passing XML around, the overhead will be minimal in comparison to the other processing.
回答5:
If you're happy with writing JNI, consider Boost.Interprocess. This will give you portable shared memory on both Linux and Windows. Remember that there's no kernel roundtrip for reading/writing shared memory.
回答6:
If it's a big chunk of data in "one" function call I would recommend JNI.
Take a look at this:Sharing output streams through a jni interface
Snippet from the article, it transfert data from c++ to java, the opposite would be also easy to do:
In all, the general strategy for sharing binary data (A/V files, images, etc.) from C with Java requires byte arrays. You create a Java byte array in C like this:
const char[] rawData = {0,1,2,3,4,5,6,7,8,9}; //Or get some raw data from somewhere
int dataSize = sizeof(rawData);
printf("Building raw data array copy\n");
jbyteArray rawDataCopy = env->NewByteArray(dataSize);
env->SetByteArrayRegion(rawDataCopy, 0, dataSize, rawData);
And pass it to Java like this:
printf("Finding callback method\n");
//Assumes obj is the Java instance that will receive the raw data via callback
jmethodID aMethodId = env->GetMethodID(env->GetObjectClass(obj),"handleData","([B)V");
if(0==aMethodId) throw MyRuntimeException("Method not found error");
printf("Invoking the callback\n");
env->CallVoidMethod(obj,aMethodId, &rawDataCopy);
you would have a Java object that looked something like this:
public class MyDataHandler {
OutputStream dataStream;
public MyDataHandler(OutputStream writeTo) { dataStream = writeTo;}
public void handleData(byte[] incomingData) { dataStream.write(incomingData); }
}
That handler would be passed to C via native method like so:
public class NativeIntegration {
public native void generateBinaryWithHandler(MyDataHandler handler);
//Here we assume response is something like a network stream
public void doCallNativeFunction(ResponseStream response) {
MyDataHandler handler = new MyDataHandler(response);
generateBinaryWithHandler(handler);
}
}
Also, you can use other technologies: CORBA, ASN.1 (ASN.1 tool), UDP or TCP
回答7:
I would use a local socket with negative acknowledgment UDP if the bit rate is going to be too high for TCP (although I would try TCP first and confirm that it is a problem). There should be minimal, if any, packets dropped if you are doing the streaming on the same machine, but the addition of a negative acknowledgement layer will take care of that case for you.
回答8:
How about using System.out and System.in?
If that isn't suitable, then Sockets is your best bet.
回答9:
If your C++ process starts the Java process, it might benefit from the inheritedChannel. Also, if the Java process is using a file, I recommend exploration of the transferTo and transferFrom methods. When doing file to file IO, these avoid needlessly tripping back and forth between user and kernel space; the same optimizations might kick in if you're using a special socket channel.
回答10:
I recommend a UDP "connection" that acknowledges every Nth packet that was received without fault, and requests a retransmission of the few packets it will miss.
回答11:
I would advise against JNI, because it is very difficult to debug. If the C++ code segfaults or throws an uncaught exception, your JVM will crash, and you will have no idea why.
来源：https://stackoverflow.com/questions/266913/efficient-data-transfer-from-java-to-c-on-windows