The piece of music chosen for the test is of a digital recording of a Hyden concerto for trumpet and orchestra. The piece has good dynamics and is rich in high frequency components. The use of harpsichord provides a subtle accompaniment in the background useful to assess the masking effects of loud foreground sounds over the background detail.
The method used for comparing the encoders was to encode the same section using the various codecs and then to decode the output back to a wave file using the appropriate decoder. The resulting wave file was then loaded into CoolEdit and a frequency analysis performed to produce the graphs in this document. While I was about it I also performed a visual inspection of the file using spectral view to assess the distribution of high frequency components.
All encoders were used in their highest quality mode and the best available decoder used to convert the file back to a wave file again. Listening tests were performed by playing the wave files rather than using an MP3 player. A pair of Senheiser HD420 headphones were used for the the listening tests. I have an AWE32 sound card in my K6-233 based machine coupled to a high quality home built 10W class A headphone amplifier. The AWE is a bit noisy but using it coupled to an external amplifier provides pretty good sound quality.
Although I consider listening tests to be far more important than measurements with instruments I do like to have some kind of scientific data to justify my personal perceptions. The purpose of these tests are therefore to try and show some correlation between perceived sound quality and measurable data. As headphones were used for the listening tests no accurate opinion can be given on the effects on stereo imaging of the various codecs.
I decided real music must be used to gather the data as white noise tests and frequency sweeps are not going to subject the codecs to the conditions they are designed to cope with. The perceptual encoding techniques used by the codecs were never meant to handle sterile test signals but rather real music and speech. The resulting graphs will clearly show how the masking effects of loud low frequency sounds are used to allow for the reduction in high frequency content.