Delta calculation between MIDIPackets do not seem to be right

Question

I am trying to read a midi file, and play all the midi events with a synthesizer. The way the synth works is, it has a circular buffer which you write midi data to, and then call GenerateSamples() on it, and it will process that midi data, and give you back the number of samples you want. I'm using AudioToolbox's music player, and have setup a MidiReadProc where I write those midi packets to a buffer, and then have a separate thread polling that buffer, and writing the data to the ring buffer, and generating the appropriate number of samples. I'm taking the delta of the timestamp of the current MIDIPacket, and the previous MIDIPacket. The result is, for the most part, the duration of all note events are wrong, and note events start at the wrong time-- which can only mean the deltas between the timestamps are wrong.. And I don't understand how that can be.

My MidiReadProc is:

static void midiReadProc(const MIDIPacketList *pktlist,
                         void *refCon,
                         void *connRefCon) {

    struct MidiData *midiData = (struct MidiData *)refCon;

    MIDIPacket *packet = (MIDIPacket *)pktlist->packet;
    for (int i = 0; i < pktlist->numPackets; i++) {
        NSUInteger channel = packet->data[0] & 0xf;
        FMInstrument *fmInstrument = midiData->fmInstruments[channel];
        [fmInstrument enqueueMidiPacket:*packet];
        packet = MIDIPacketNext(packet);
    }
}

And FMInstrument's enqueueMidiPacket: does:

-(void)enqueueMidiPacket:(MIDIPacket)packet {
    _midiPackets[_midiPacketWriteIndex] = packet;
    _midiPacketWriteIndex += 1;
}

Meanwhile, when FMInstrument is initialized, it kicks off a poller thread:

-(instancetype)init {
    if (self = [super init]) {
        _ringBuffer       = new RingBuffer();
        _synthUnit        = new SynthUnit(_ringBuffer);
        _maxBufferIndex   = (int)([self sampleRate] * 45) - 1;
        _generatedSamples = (short *)calloc(_maxBufferIndex + 1, sizeof(short));
        _midiPackets      = (MIDIPacket *)calloc(1024, sizeof(MIDIPacket));
        _bufferReadIndex  = 0;
        _bufferWriteIndex = 0;
        _midiPacketReadIndex  = 0;
        _midiPacketWriteIndex = 0;
        _lastMidiPacket       = {0};
        mach_timebase_info_data_t sTimebaseInfo;
        mach_timebase_info(&sTimebaseInfo);

        _nanoSecondsToSeconds = sTimebaseInfo.numer / sTimebaseInfo.denom / 1000000000.0;

        [NSThread detachNewThreadSelector:@selector(startPolling) toTarget:self withObject:nil];
    }
    return self;
}

and the polling method:

-(void)startPolling {
    while ([self shouldPoll]) {
        @autoreleasepool {
            if (_midiPacketWriteIndex <= _midiPacketReadIndex) continue;
            MIDIPacket midiPacket     = _midiPackets[_midiPacketReadIndex];
            MIDIPacket lastMidiPacket = _lastMidiPacket;

            _ringBuffer->Write(midiPacket.data, midiPacket.length);
            BOOL isFirstEvent = !_lastMidiPacket.timeStamp;
            _lastMidiPacket = midiPacket;
            if (isFirstEvent) continue;

            _midiPacketReadIndex += 1;
            double delta = (midiPacket.timeStamp * _nanoSecondsToSeconds) - (lastMidiPacket.timeStamp *_nanoSecondsToSeconds);

            int numberOfSamplesToGenerate = ceil([self sampleRate] * delta);
            short *tempBuffer = (short *)malloc(sizeof(short) * numberOfSamplesToGenerate);
            _synthUnit->GetSamples(numberOfSamplesToGenerate, tempBuffer);

            for (int i = 0; i < numberOfSamplesToGenerate && _bufferWriteIndex <= _maxBufferIndex; i++) {
                _generatedSamples[_bufferWriteIndex] = tempBuffer[i];
                _bufferWriteIndex += 1;
            }

            free(tempBuffer);
            [NSThread sleepForTimeInterval:0.05f];
        }
    }
}

So I am assuming, these two lines must be incorrect:

double delta = (midiPacket.timeStamp * _nanoSecondsToSeconds) - (lastMidiPacket.timeStamp *_nanoSecondsToSeconds);
int numberOfSamplesToGenerate = ceil([self sampleRate] * delta);

Can anyone tell me what I am doing wrong?