Low latency input/output AudioQueue

问题

I have two iOS AudioQueues - one input that feeds samples directly to one output. Unfortunately, there is an echo effect that is quite noticeable :(

Is it possible to do low latency audio using AudioQueues or do I really need to use AudioUnits? (I have tried the Novocaine framework that uses AudioUnits and here the latency is much smaller. I've also noticed this framework seems to use less CPU resources. Unfortunately, I was unable to use this framework in my Swift project without major changes to it.)

Here are some extracts of my code, which is mainly done in Swift, except those callbacks which needs to be implemented in C.

private let audioStreamBasicDescription = AudioStreamBasicDescription(
    mSampleRate: 16000,
    mFormatID: AudioFormatID(kAudioFormatLinearPCM),
    mFormatFlags: AudioFormatFlags(kAudioFormatFlagsNativeFloatPacked),
    mBytesPerPacket: 4,
    mFramesPerPacket: 1,
    mBytesPerFrame: 4,
    mChannelsPerFrame: 1,
    mBitsPerChannel: 32,
    mReserved: 0)

private let numberOfBuffers = 80
private let bufferSize: UInt32 = 256

private var active = false

private var inputQueue: AudioQueueRef = nil
private var outputQueue: AudioQueueRef = nil

private var inputBuffers = [AudioQueueBufferRef]()
private var outputBuffers = [AudioQueueBufferRef]()
private var headOfFreeOutputBuffers: AudioQueueBufferRef = nil

// callbacks implemented in Swift
private func audioQueueInputCallback(inputBuffer: AudioQueueBufferRef) {
    if active {
        if headOfFreeOutputBuffers != nil {
            let outputBuffer = headOfFreeOutputBuffers
            headOfFreeOutputBuffers = AudioQueueBufferRef(outputBuffer.memory.mUserData)
            outputBuffer.memory.mAudioDataByteSize = inputBuffer.memory.mAudioDataByteSize
            memcpy(outputBuffer.memory.mAudioData, inputBuffer.memory.mAudioData, Int(inputBuffer.memory.mAudioDataByteSize))
            assert(AudioQueueEnqueueBuffer(outputQueue, outputBuffer, 0, nil) == 0)
        } else {
            println(__FUNCTION__ + ": out-of-output-buffers!")
        }

        assert(AudioQueueEnqueueBuffer(inputQueue, inputBuffer, 0, nil) == 0)
    }
}

private func audioQueueOutputCallback(outputBuffer: AudioQueueBufferRef) {
    if active {
        outputBuffer.memory.mUserData = UnsafeMutablePointer<Void>(headOfFreeOutputBuffers)
        headOfFreeOutputBuffers = outputBuffer
    }
}

func start() {
    var error: NSError?
    audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, withOptions: .allZeros, error: &error)
    dumpError(error, functionName: "AVAudioSessionCategoryPlayAndRecord")
    audioSession.setPreferredSampleRate(16000, error: &error)
    dumpError(error, functionName: "setPreferredSampleRate")
    audioSession.setPreferredIOBufferDuration(0.005, error: &error)
    dumpError(error, functionName: "setPreferredIOBufferDuration")

    audioSession.setActive(true, error: &error)
    dumpError(error, functionName: "setActive(true)")

    assert(active == false)
    active = true

    // cannot provide callbacks to AudioQueueNewInput/AudioQueueNewOutput from Swift and so need to interface C functions
    assert(MyAudioQueueConfigureInputQueueAndCallback(audioStreamBasicDescription, &inputQueue, audioQueueInputCallback) == 0)
    assert(MyAudioQueueConfigureOutputQueueAndCallback(audioStreamBasicDescription, &outputQueue, audioQueueOutputCallback) == 0)

    for (var i = 0; i < numberOfBuffers; i++) {
        var audioQueueBufferRef: AudioQueueBufferRef = nil
        assert(AudioQueueAllocateBuffer(inputQueue, bufferSize, &audioQueueBufferRef) == 0)
        assert(AudioQueueEnqueueBuffer(inputQueue, audioQueueBufferRef, 0, nil) == 0)
        inputBuffers.append(audioQueueBufferRef)

        assert(AudioQueueAllocateBuffer(outputQueue, bufferSize, &audioQueueBufferRef) == 0)
        outputBuffers.append(audioQueueBufferRef)

        audioQueueBufferRef.memory.mUserData = UnsafeMutablePointer<Void>(headOfFreeOutputBuffers)
        headOfFreeOutputBuffers = audioQueueBufferRef
    }

    assert(AudioQueueStart(inputQueue, nil) == 0)
    assert(AudioQueueStart(outputQueue, nil) == 0)
}

And then my C-code to set up the callbacks back to Swift:

static void MyAudioQueueAudioInputCallback(void * inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer, const AudioTimeStamp * inStartTime,
                                   UInt32 inNumberPacketDescriptions, const AudioStreamPacketDescription * inPacketDescs) {
    void(^block)(AudioQueueBufferRef) = (__bridge void(^)(AudioQueueBufferRef))inUserData;
    block(inBuffer);
}

static void MyAudioQueueAudioOutputCallback(void *inUserData, AudioQueueRef inAQ, AudioQueueBufferRef inBuffer) {
    void(^block)(AudioQueueBufferRef) = (__bridge void(^)(AudioQueueBufferRef))inUserData;
    block(inBuffer);
}

OSStatus MyAudioQueueConfigureInputQueueAndCallback(AudioStreamBasicDescription inFormat, AudioQueueRef *inAQ, void(^callback)(AudioQueueBufferRef)) {
    return AudioQueueNewInput(&inFormat, MyAudioQueueAudioInputCallback, (__bridge_retained void *)([callback copy]), nil, nil, 0, inAQ);
}

OSStatus MyAudioQueueConfigureOutputQueueAndCallback(AudioStreamBasicDescription inFormat, AudioQueueRef *inAQ, void(^callback)(AudioQueueBufferRef)) {
    return AudioQueueNewOutput(&inFormat, MyAudioQueueAudioOutputCallback, (__bridge_retained void *)([callback copy]), nil, nil, 0, inAQ);
}

回答1:

After a good while I found this great post using AudioUnits instead of AudioQueues. I just ported it to Swift and then simply added:

audioSession.setPreferredIOBufferDuration(0.005, error: &error)

回答2:

If you're recording audio from a microphone and playing it back within earshot of that microphone, then due to the audio throughput not being instantaneous, some of your previous output will make it into the new input, hence the echo. This phenomenon is called feedback.

This is a structural problem, so changing the recording API won't help (although changing your recording/playback buffer sizes will give you control over the delay in the echo). You can either play back the audio in such a way that the microphone can't hear it (e.g. not at all, or through headphones) or go down the rabbit hole of echo cancellation.

来源：https://stackoverflow.com/questions/30011730/low-latency-input-output-audioqueue