AudioKit - How to get Real Time floatChannelData from Microphone?

Question

I'm new to Audiokit and I'm trying to do some real-time digital signal processing on input audio from the microphone.

I know the data I want is in AKAudioFile's FloatChannelData, but what if I want to obtain this in real-time? I'm currently using AKMicrophone, AKFrequencyTracker, AKNodeOutputPlot, AKBooster and I'm plotting the tracker's amplitude data. However, that data is not the same as the audio signal (as you know, it's the RMS). Is there any way I can obtain the signal's Float data from the mic? Or even from the AKNodeOutputPlot? I just need read-access.

AKSettings.audioInputEnabled = true
mic = AKMicrophone()
plot = AKNodeOutputPlot(mic, frame: audioInputPlot.bounds)
tracker = AKFrequencyTracker.init(mic)
silence = AKBooster(tracker,gain:0)
AudioKit.output = silence
AudioKit.start()

The creator of recommends here:

AKNodeOutputPlot works, its one short file. You're basically just tapping the node and grabbing the data.

How would this work in my viewController if if have an instance of plot (AKNodeOutputPlot), mic(AKMicrophone) and want to output those values to a label?

Answer 1

Use a tap on which ever node you want to get the data out from. I used AKNodeOutputPlot in my quote above because its is fairly straightforward, just using that data as input for a plot, but you could take the data and do whatever with it. In this code (from AKNodeOutputPlot):

internal func setupNode(_ input: AKNode?) {
    if !isConnected {
        input?.avAudioNode.installTap(
            onBus: 0,
            bufferSize: bufferSize,
            format: nil) { [weak self] (buffer, _) in

                guard let strongSelf = self else {
                    AKLog("Unable to create strong reference to self")
                    return
                }
                buffer.frameLength = strongSelf.bufferSize
                let offset = Int(buffer.frameCapacity - buffer.frameLength)
                if let tail = buffer.floatChannelData?[0] {
                    strongSelf.updateBuffer(&tail[offset], withBufferSize: strongSelf.bufferSize)
                }
        }
    }
    isConnected = true
}

You get the buffer data in real time. Here we just send it to "updateBuffer" where it gets plotted, but instead of plotting you'd do something else.

Answer 2

To complete Aurelius Prochazka answer:

To record the audio flowing through a node, you need to attach a tape to it. A tape is just a closure which get called each time a buffer is available.

Here is a sample code you can reuse in your own class:

var mic = AKMicrophone()

func initMicrophone() {

  // Facultative, allow to set the sampling rate of the microphone
  AKSettings.sampleRate = 44100

  // Link the microphone note to the output of AudioKit with a volume of 0.
  AudioKit.output = AKBooster(mic, gain:0)

  // Start AudioKit engine
  try! AudioKit.start()

  // Add a tape to the microphone
  mic?.avAudioNode.installTap(
      onBus: audioBus, bufferSize: 4096, format: nil // I choose a buffer size of 4096
  ) { [weak self] (buffer, _) in //self is now a weak reference, to prevent retain cycles

      // We try to create a strong reference to self, and name it strongSelf
      guard let strongSelf = self else {
        print("Recorder: Unable to create strong reference to self #1")
        return
      }

      // We look at the buffer if it contains data
      buffer.frameLength = strongSelf.bufferSize
      let offset = Int(buffer.frameCapacity - buffer.frameLength)
      if let tail = buffer.floatChannelData?[0] {
        // We convert the content of the buffer to a swift array
        let samples = Array(UnsafeBufferPointer(start: &tail[offset], count: 4096))
        strongSelf.myFunctionHandlingData(samples)
      }
  }

  func myFunctionhandlingData(data: [Float]) {
    // ...
  }

Be careful to use DispatchQueue or an other synchronisation mechanism if you need a to interact on this data between different threads. In my case I do use :

DispatchQueue.main.async { [weak self]  in
  guard let strongSelf = self else {
    print("Recorder: Unable to create strong reference to self #2")
    return
  }
  strongSelf.myFunctionHandlingData(samples)
}

so that my function run in the main thread.