The WaveFormat of your sampleProvider class should be IEEE. Please do a sampleProvider.WaveFormat.ToString() and tell me what you've got.
You can go back fom ISampleProvider to IWaveProvider using SampleToWaveProvider in order to play the audio. (n.b. in the very latest code there is SampleToWaveProvider16, which will also put it back to 16 bit PCM).