How to process audio stream in realtime

问题

I have a setup with a raspberry pi 3 running latest jessie with all updates installed in which i provide a A2DP bluetooth sink where i connect with a phone to play some music.

Via pulseaudio, the source (phone) is routed to the alsa output (sink). This works reasonably well.

I now want to analyze the audio stream using python3.4 with librosa and i found a promising example using pyaudio which got adjusted to use the pulseaudio input (which magically works because its the default) instead of a wavfile:

"""PyAudio Example: Play a wave file (callback version)."""

import pyaudio
import wave
import time
import sys
import numpy

# instantiate PyAudio (1)
p = pyaudio.PyAudio()

# define callback (2)
def callback(in_data, frame_count, time_info, status):
    # convert data to array
    data = numpy.fromstring(data, dtype=numpy.float32)
    # process data array using librosa
    # ...
    return (None, pyaudio.paContinue)

# open stream using callback (3)
stream = p.open(format=p.paFloat32,
                channels=1,
                rate=44100,
                input=True,
                output=False,
                frames_per_buffer=int(44100*10),
                stream_callback=callback)

# start the stream (4)
stream.start_stream()

# wait for stream to finish (5)
while stream.is_active():
    time.sleep(0.1)

# stop stream (6)
stream.stop_stream()
stream.close()
wf.close()

# close PyAudio (7)
p.terminate()

Now while the data flow works in principle, there is a delay (length of buffer) with which the stream_callback gets called. Since the docs state

Note that PyAudio calls the callback function in a separate thread.

i would have assumed that while the callback is worked on, the buffer keeps filling in the mainthread. Of course, there would be an initial delay to fill the buffer, afterwards i expected to get synchronous flow.

I need a longer portion in the buffer (see frames_in_buffer) for librosa to be able to perfom analysis correctly.

How is something like this possible? Is it a limitation of the software-ports for the raspberry ARM?

I found other answers, but they use the blocking I/O. How would i wrap this into a thread so that librosa analysis (which might take some time) does not block the buffer filling?

This blog seems to fight performance issues with cython, but i dont think the delay is a performance issue. Or might it? Others seem to need some ALSA tweaks but would this help while using pulseaudio?

Thanks, any input appreciated!

来源：https://stackoverflow.com/questions/42625286/how-to-process-audio-stream-in-realtime

标签

python-3.x

raspberry-pi3

pyaudio

pulseaudio