Have you tried running it through a spectrogram (function spectrogram in MATLAB) to identify what is happening?
I don't know what algorithms you use, without that information, we can't say what is going wrong. What alarms me is that your second third harmonic (second peak in the plot) is much larger than your second harmonic (first peak in the plot).
Are you sure you have all the sampling right: i.e. your DFT only has frequencies up to half the sampling frequency (both positive and negative frequency range)? Also: how do you suppress any transient part of your signal?
