PortAudio real-time audio processing for continuous input stream

问题

I am using PortAudio to implement a real-time audio processing.

My primary task is to acquire data from mic continuously and provide 100 samples for processing (each FRAME = 100 samples at a time) to some other processing thread.

Here is my callback collecting 100 samples each time on a continuous basis -

static int paStreamCallback( const void* input, void* output,
    unsigned long samplesPerFrame,
    const PaStreamCallbackTimeInfo* timeInfo,
    PaStreamCallbackFlags statusFlags,
    void* userData ) {

    paTestData *data = (paTestData*) userData;
    const SAMPLE *readPtr = (const SAMPLE*)input;   // Casting input read to valid input type SAMPLE (float)
    unsigned long totalSamples = TOTAL_NUM_OF_SAMPLES ; // totalSamples = 100 here

    (void) output;
    (void) timeInfo;
    (void) statusFlags;

    static int count = 1;

    if(data->sampleIndex < count * samplesPerFrame){
        data->recordedSample[data->sampleIndex] = *readPtr;
        data->sampleIndex++;
    }

    else if(data->sampleIndex ==  count * samplesPerFrame){

        processSampleFrame(data->recordedSample);
        count++;
        finished = paContinue;
    }

    return finished;
}

My `main function -

int main(void){

// Some Code here

data.sampleIndex = 0;
data.frameIndex = 1;

numBytes = TOTAL_NUM_OF_SAMPLES * sizeof(SAMPLE);
data.recordedSample = (SAMPLE*)malloc(numBytes);

for(i=0; i < TOTAL_NUM_OF_SAMPLES; i++)
    data.recordedSample[i] = 0;

// Open Stream Initialization

err = Pa_StartStream(stream);

/* Recording audio here..Speak into the MIC */
printf("\nRecording...\n");
fflush(stdout);

while((err = Pa_IsStreamActive(stream)) == 1)
    Pa_Sleep(10);

    if(err < 0)
            goto done;

    err = Pa_CloseStream(stream);

// Some more code here
}

Sending each Frame of 100 samples to processSampleFrame.

void processSampleFrame(SAMPLE *singleFrame){

    // Free buffer of this frame
    // Processing sample frame here
}

The challenge is that I need to implement a way in which the time processSampleFrame is processing the samples, my callBack should be active and keep acquiring the next Frame of 100 samples and (may be more depending upon the processing time of processSampleFrame).

Also the buffer should able to free itself of the frame so sooner it has passed it to processSampleFrame.

EDIT 2 :

I tried implementing with PaUtilRingBuffer that PortAudio provides. Here is my callback.

printf("Inside callback\n");
paTestData *data = (paTestData*) userData;

ring_buffer_size_t elementsWritable = PaUtil_GetRingBufferWriteAvailable(&data->ringBuffer);
ring_buffer_size_t elementsToWrite = rbs_min(elementsWritable, (ring_buffer_size_t)(samplesPerFrame * numChannels));

const SAMPLE *readPtr = (const SAMPLE*)input;
printf("Sample ReadPtr = %.8f\n", *readPtr);
(void) output;      // Preventing unused variable warnings
(void) timeInfo;
(void) statusFlags;

data->frameIndex += PaUtil_WriteRingBuffer(&data->ringBuffer, readPtr, elementsToWrite);

return paContinue;

And main :

int main(void){

    paTestData data;    // Object of paTestData structure

    fflush(stdout);

    data.frameIndex = 1;

    long numBytes = TOTAL_NUM_OF_SAMPLES * LIMIT;
    data.ringBufferData = (SAMPLE*)PaUtil_AllocateMemory(numBytes);
    if(PaUtil_InitializeRingBuffer(&data.ringBuffer, sizeof(SAMPLE), ELEMENTS_TO_WRITE, data.ringBufferData) < 0){
        printf("Failed to initialise Ring Buffer.\n");
        goto done;

    err = Pa_StartStream(stream);

    /* Recording audio here..Speak into the MIC */
    printf("\nRecording samples\n");
    fflush(stdout);

    while((err = Pa_IsStreamActive(stream)) == 1)
        Pa_Sleep(10);

    if(err < 0)
                goto done;

        err = Pa_CloseStream(stream);

    // Error Handling here
}

PaTestData Structure :

typedef struct{

    SAMPLE *ringBufferData;
    PaUtilRingBuffer ringBuffer;    
    unsigned int frameIndex;
}
paTestData;

I am facing the same issue of seg-fault after successful acquisition for the allocated space because of not being able to use any free in the callback as suggested by PortAudio documentation.

Where can I free the buffer of the allocated frame given to the processing thread. May be a method of obtaining a thread-synchronization can be really useful here. Your help is appreciated.

回答1:

Example code of processing audio input:

#define FRAME_SIZE                 1024
#define CIRCULAR_BUFFER_SIZE       (FRAME_SIZE * 4)

float buffer[CIRCULAR_BUFFER_SIZE];

typedef struct {
     int read, write;
     float vol;
} paData;

static int paStreamCallback(const void* input, void* output,
                            unsigned long samplesPerFrame,
                            const PaStreamCallbackTimeInfo* timeInfo,
                            PaStreamCallbackFlags statusFlags,
                            void* userData) {

    paData *data = (paData *)userData;
    // Write input buffer to our buffer: (memcpy function is fine, just a for loop of writing data
    memcpy(&buffer[write], input, samplesPerFrame); // Assuming samplesPerFrame = FRAME_SIZE

    // Increment write/wraparound
    write = (write + FRAME_SIZE) % CIRCULAR_BUFFER_SIZE;

    // dummy algorithm for processing blocks:
    processAlgorithm(&buffer[read]);

    // Write output
    float *dummy_buffer = &buffer[read]; // just for easy increment
    for (int i = 0; i < samplesPerFrame; i++) {
         // Mix audio input and processed input; 50% mix
         output[i] = input[i] * 0.5 + dummy_buffer[i] * 0.5;
    }

    // Increment read/wraparound
    read = (read + FRAME_SIZE) % CIRCULAR_BUFFER_SIZE;

    return paContinue;
}

int main(void) {
     // Initialize code here; any memory allocation needs to be done here! default buffers to 0 (silence)
     // initialize paData too; write = 0, read = 3072
     // read is 3072 because I'm thinking about this like a delay system.
}

Take this with a grain of salt; Obviously better ways of doing this but it can be used as a starting point.

来源：https://stackoverflow.com/questions/44645466/portaudio-real-time-audio-processing-for-continuous-input-stream