I have a networking Linux application which receives RTP streams from multiple destinations, does very simple packet modification and then forwards the streams to the final
I would look into a thread pool for this application.
http://threadpool.sourceforge.net/
Allow the thread pool to manage your threads and the queue.
You can tweak the maximum and minimum number of threads used based on performance profiling later.