popen pipe slows down other threads

问题

I have problem with my multithread application. When at one thread executing synchronous popen() command - other application threads are slow down significantly. Thread with popen() execute ffmpeg, that generates high load.

Normally, other threads execution time is 0.0007 ms. And when popen is used, some threads are increasing there execution time up to 14-20 seconds.

How to solve this problem?

System is FreeBSD 6.4

    FILE *pipe;
    char buff[512];
    if ( !(pipe = popen( command.c_str(), "r")) )
    { // if pipe is NULL
        return false;
    }

    while ( fgets(buff, sizeof(buff), pipe) != NULL )
    {
        ptr_output->append(buff);
    }

here is new code of popen can that does NOT help: Correct Code - Non-blocking pipe with popen

回答1:

fgets is a blocking read, so while the thread above is waiting for data to be read from the pipe, the other threads are blocked. You will want to use select/poll for with a file descriptor to see if you have data on the pipe before you issue a read. That way, you can preempt this thread, and let other threads run doing useful work.

回答2:

What is the relationship between the different threads? If they depend on each other, meaning they send data back and forth, then if one thread goes slower, it makes sense that the others would as well.

Something else to consider is how the thread that is executing ffmpeg affects the rest of the system. For example, if its a single-core CPU and that particular thread is generating a high CPU load, then that would leave less cycles for the rest of the threads, thus slowing them down. Granted, a change from 0.0007 ms to 14 - 20 seconds is indeed extreme!

Are there any other resources shared (stdin, mutexes, etc) between the threads that the high load thread could be abusing (holding/locking too long), thus causing starvation for the other threads?

Additionally, I would suggest profiling the application (or at least some of the threads) to see why it is so much slower. Im almost positive that you'll find out that some threads are blocked waiting for a common resource, like a mutex or something similar.

If this is linux, here are 2 Stack Overflow questions that may help:

C++ Tips for code optimization on ARM devices
How to profile pthread mutex in linux?

来源：https://stackoverflow.com/questions/10960622/popen-pipe-slows-down-other-threads

标签

c++

multithreading

pthreads

popen