mobanmarketWhen we launched Stereophile‘s website at the end of 1997, we decided that we would not reprint the magazine’s most popular features, including the biannual “Recommended Components” listings and Michael Fremer’s monthly “Analog Corner” column. We were concerned that doing so would cannibalize magazine sales. As it turned out, we were wrong—and so the latest “Recommended Components” is available on our free-access website day-and-date with the publication of the April and October issues in which it appears. And starting with Mikey’s very first “Analog Corner,” from July 1995, I have been posting his column on our AnalogPlanet.com website.
At the time of writing, I had just finished HTML-encoding Michael’s October 2004 column, and fell over an exchange between him and popular technology writer Walter Mossberg, at that time a contributor to the Wall Street Journal. The essence of Mossberg’s argument was that people who say that there is better sound than listening to MP3s are snobs—that most of his readers listened to music on computers through plastic speakers and “love—love” listening that way.
Michael responded that all people had to do was to listen for themselves. This, of course, was the rock on which Stereophile was founded when the late J. Gordon Holt published his first issue, in November 1962. And Gordon presaged Mossberg’s point by almost two decades when he wrote in 1987: “As the person who ‘invented’ subjective testing, I have followed with great interest the many articles in the mainstream audio press which purport to prove that none of us can really hear all the differences we claim to hear, particularly those between amplifiers.”
Differences between amplifiers? I flashed back on an amplifier listening test in which I was involved in 1978. Before becoming an audio journalist, I’d been a professional musician, and before that I worked in scientific research. My formal education had been in the sciences, both at the bachelor and postgraduate levels, and while I wasn’t a “hardcore” objectivist back then, I was skeptical of those who wrote about enormous differences in sound quality. Martin Colloms had organized a test for Hi-Fi News magazine in which we would try to identify three amplifiers, including a solid-state Quad and a tubed Michaelson & Austin, by listening to them under blind conditions. Martin’s test methodology seemed, to this erstwhile scientist, to be beyond reproach, and when the analysis of the test results showed that there was no statistically significant audible difference between the amplifiers, I bought a Quad 405.
Worst audio purchase I ever made. I stopped getting any musical satisfaction from my system. It wasn’t until I replaced the Quad with an EL34-tube–equipped Michaelson & Austin TVA-10 that my musical enjoyment returned to its previous level.
All I could conclude was that the blind-test protocol itself had become what scientists call an “interfering variable”—that the conditions of the test were too far removed from how we listen to music through our systems to give meaningful results. I have participated in a large number of blind tests since 1978, and have found that, even when a real difference exists, it is very difficult to produce anything but statistically null results—what are called “false negatives.” In the end, I decided that if so-called “objective” testing lets you down, it’s best to follow J. Gordon Holt’s strategy and judge equipment by how it sounds playing real music in real time.
This has pitfalls of its own. When such tests are poorly performed, they can produce “false positives”—ie, the listener concludes that there is a difference when none exists. In an article in the November 1980 issue of Hi-Fi News & Record Review, since republished on this website, I wrote: “The problem confronting the magazine reviewer when organizing the necessary listening tests to accompany/reinforce the measured behavior of a device under test is complex. . . . Unlike the reaction of an oscilloscope, that of a listener involves interaction: what he is hearing; what he had been expecting to hear; the identity of the equipment; the emotional effect of the music program; the emotional effect of other competing stimuli (a recent cup of coffee, a not-so-recent visit to the toilet); the apparent expectations of his fellow listeners; the ultimate purpose of the test; the desire for self-consistency and hence self-esteem; all these can—but needn’t always—color the listener’s assessment. Obviously, this will affect the reliability of any conclusion, both when used to predict the same listener’s reaction to the same piece of equipment, and when used to predict other people’s reactions.”
I read enough reviews published elsewhere where I do indeed doubt the reliability of the writer’s conclusions. As Stereophile‘s editor, therefore, I work hard to ensure that, when you read our reviewers’ descriptions of sound quality, those descriptions are both repeatable and transportable. We include enough information in our reviews about the music that was used in the testing that you, the reader, can use the same music to see if you agree with our findings. When possible, I visit the members of our team and listen to their systems, where inserting the product being reviewed is the only change. That listening may seem informal and collegial—as it was with the Revel Salon2 speakers Jim Austin writes about in this issue—but it is basically a test of the tester. I want to be able to hear for myself what the reviewer describes. And when a reviewer’s auditioning of a product raises questions, I listen to it myself in my own system.
Stereophile‘s readers deserve no less.
Click Here: factory direct women products