Are you able to tell if any clipping occured (i.e. did the summed audio go over 1.0 at any point)? The conversion back to 16 bit ought to clip to plus or minus 1.0 before multiplying up by 32,768 to get back to a 16 bit number.
Also, a good way to debug this kind of problem is to save your inputs to WAV files, and then pass those WAV files through the mixer into another WAV file. Then you can look at the results in an audio editor and get a good idea of what the problem is.
Mark