What is the standard for judging a systems sound?


It is often said in these threads that this hobby is all about the music. That live music is the only meaningful standard for comparison when determining the quality of a stereo system. While these words sound good, are they really true?

A violin should sound like a violin, a flute should sound like a flute, and a guitar should sound like a guitar. Many purists will immediately say that amplified/electronic music cannot be used as a standard since a listener can never really know what the intention of the musician was when he/she recorded it, and what that sound should be.

Even something as simple as an electric guitar has multiple settings from which to choose. Electronic keyboards have hundreds of possible voices, so how does the poor audiophile know how the tone was supposed to sound?

These are valid concerns. Back to the purists!
“That’s why only unamplified classical music can be used as a standard!!!” On face value that looks like an acceptable statement. Consider some facts though. In my immediate family we a have several musicians who play a few different instruments. We have an electric piano (due to a distinct lack of room for a baby grand), acoustic guitar, Fender Stratocaster electric guitar, a nickel plated closed hole flute, a silver plated open hole flute, a viola, and a cello.

I have a fairly good idea how each of these instruments sound. One comment I must make immediately is that they sound a little different in different rooms. Another comment, which demands attention: when I bought my first flute I knew nothing about flutes. I began fooling around with it and enjoyed the sound. I liked it so much a bought a better, as mentioned silver open-hole flute. This flute sounded much better than the first flute. The tone was richer (the only words I can think of to describe the difference).

The reason for that background information is to show that the same instruments in different room’s sound different, AND different models of the same instrument have a much different sound!

If we audiophiles are using live unamplified music as a standard there are still several important issues, which must be addressed. How do we really know what we are hearing? What instrument is the musician playing? Was that a Gemeinhardt or Armstrong Flute. What are the sonic characteristics of the specific instrument. Stradivarius violins sound different than other violins, if they didn’t people would not be willing to pursue them so aggressively. Better instruments (theoretically anyway) sound better than lesser instruments. The point here is that different versions of the same instrument sound different.

I have seen the same music reproduced in different settings. I have heard string quartets play in a garden in Vienna. I have heard the Pipe Organ in Stephan’s Dom. I have heard Rock and Roll in arenas and Performing Arts Centers. I have heard jazz played in small one room clubs, not to mention the above listed instruments played in the house.

Each one of these venues sounds different from the other.

When I am listening to a selection of music at home, how do I know how it is supposed to sound? None of the LPs sounds like any of the particular places I have heard live music, while none of those places sounded like any other either.

There is no standard by which to judge the quality of live music since no two venues sound alike. If everyone were to go to the Royal Opera House in Covent Garden and hear Tchaikovsky’s Symphony No. 6 would everyone hear the same thing? Even if they did, and that one concert became the standard by which all other recorded music was judged, would that be translatable to allow the judging of all other music?

I have never heard a cello reproduced as well as my sons playing in the living room. I have never heard better flute players sound better than my own terrible playing at home.

So what do we audiophiles really use as the standard by which recorded music can be judged?
128x128nrchy
Nrchy: The way to know the difference between the mastertape and the original event would be to listen at the mic position during the original event - except that you can never know exactly what was captured on the mastertape without putting it through some kind of playback system that will distort it. And that includes the original studio's monitoring system, which may be no better than a good home system at representing the mastertape signal fed into it...

You would, however, gain insight into what the mastertape engineers and producers heard to work with.
This is a fascinating thread. It makes me wonder how equipment reviewers are able to describe the sound of any particular piece of euipment and what good, better, best means in the reviewing context.
Pubul57, despite the rhetoric about the standard being unamplified live acoustic music - a standard that (within limits) is still useful and admirable, and which I understand aspiring to - I think that in actuality, most gear is essentially reviewed in relation to other gear the reviewer has experience with. Which makes perfect sense to me, since the different sounds of components will all be more similar to one another than they will be to real live music (a large part of the reason for which has to do with the recording process of the source material fed them, BTW).
Zaikesman you are absoluely right about reviewers, both here are in publications using other gear as the standard. I don't think anyone's reference system is a valid comparison by which to measure new equipment.

No one else knows what your, my, or John Atkinson's system sounds like in the appropriate listening room. If I tell you that my new Klyne pre-amp sounds much more like live music than my old Krell pre-amp what good does that do anyone? I could be completely right and still not be making a statement that does anyone else a bit of good. For other gear to be a valid reference point everyone has to be familiar with the gear being used as a standard. Then they need to be familiar with the cabling and room in which the equipment is heard. No two rooms sound alike.

Live music is not a static target, but neither is anyone's refernce system. Zaikesman, how long has your system been the same? We neurotic audiophiles are in a constant state of flux. My system went unchanged for about a year, in the last month I bought a new pre-amp, and I'm in the process of buying new speaker cable. Nothing that changes so much can be helpful to others who do not hear that system. In the case of reviewers their system changes from week to week. How can any of them be used as a refernce. They don't even know what their systems sound like. They just know the latest interation of that system.

Zaikesman, I don't know that we really disagree in the essence of things. We both want our systems to be as good as reasonably possible, it's just that we have trouble locating a standard. I'm sure your mastertapes don't sound the same at home as they do at the mixing board or as the event in the studio. If you could actually use mastertapes you would definately have a leg up on the rest of us, but where do we stand in relation to the cello, flute or piano I hear at home?
Nrchy, I think I agree with your point about the pitfalls in using reference
systems as a point of comparison. But if you follow this line of thinking
to its logical conclusion, then it means virtually all of the advice given
within these forums regarding the sound of components is worthless.
You haven't heard my system in my room and I haven't heard yours.
What valid conclusions about sound quality can we really draw from the
words others have typed into their PCs?
To partially answer both Nrchy's and Onhwy61's questions (How long has your system been the same? What valid conclusions can we draw from the observations of others?) as they regard myself, here's some news FWIW:

Yesterday I bought an amplifier from a local Agonner, paid cash and picked it up. I did this not because I necessarily wanted a new amplifier, but because I've been planning on doing some modifications to my current amps (capacitor upgrade) and wanted a spare. However, I do confess to having another, additional motivation. My regular amps are VTL MB-185 tubed monoblocks, but for a while I have been somewhat curious to see how I would react to a modestly-priced solid-state stereo amp - like the ones I used have a few years back before I got my previous C-J tubed stereo amp, only more up-to-date. Call it a reality-check.

The amp I got is a McCormack DNA-125. My reference amps cost about $5K new, the DNA-125 about $1.7K new. Truthfully, I am still only warming this thing up, as I literally inserted it into the system less than 12 hours ago. But already I can tell there are some things it does at least as well, if not better (or merely as well but differently) as my reference. Based on my past experiences, I was in no way expecting to be seriously tempted by this amp, and probably planned to sell it once my mods were done. However, I admit that part of the reason I'm auditioning something else now, before I do the mods to the VTL's, is to confirm in my own mind whether my reference should be a 'keeper' that I sink more money into, and also to double-check my overall commitment to tubed power by bringing in something solid-state. That the comparison could be called 'unfair' from a price standpoint (the output power isn't very different between the two into 4 ohms) I'll just have to let slide, since I wasn't going to drop much more than I did for a presumed temporary experiment.

But this little McCormack is obviously a very fine performer, maybe even 'good enough' - if we can ever admit to such a thing in this hobby - and I am going to enjoy listening to it for a while. Anyway, to get to the question touched upon by this thread, I just returned from rereading Jim Merod's review of this amp, and then going to audioreview.com, where there are 35 user reviews, who gave it an average of a perfect 5 out of 5 rating. And let me tell you, both Merod's piece and almost all of the user reviews are, I think at this early stage, dead-on acccurate in their assessment of this amp.

Now, this may not mean much in isolation, especially since it kind of surprises me, as I routinely find myself disagreeing with reviews both professional and amatuer. And, of course, I have a ways to go with this amp before coming to any final conclusions - there are still things I suspect my reference does better, although in terms of value for the money I likely have some thinking to do. But clearly, this McCormack is doing many things seemingly 'right', and everyone I've read about it so far thinks so too (next stop, Agon archives). That's got to mean something I believe ; that kind of consensus, jibing as it does with my own tentative experience, cannot be easily dissmissed as devoid of significance, at least not by me...
The last thing that I thought I would ever do is feel like I have to come to the defense of audio reviewers. The truth of the matter is that although audio reviewers are oftentimes easy targets for criticism/ridicule, wether we want to admit it or not, there are in fact a few truly talented ones. Have you guys read the work of people like Jon Nork, Harry Pearson and others, over the years? I don't know about you guys, but I have found their commentary extremely useful. They helped develop a language to describe what they hear that has changed the way that we all talk about the hobby. I frankly don't find it all that difficult to get a sense of what the best reviewers are trying to say about certain components, eventhough their systems are in constant state of flux, since they (the best ones) usually make reference to the sound of the real thing in a way that to me, makes sense. The same way that we need to attend a lot of live music performances to really be able to build a good personal reference, by familiarizing ourselves with the language and yes, biases of these guys, we can get some really useful info about components. I find the across-the-board cynicism directed at reviewers unjustified and counterproductive. Are there a lot of hacks out there? You bet! So what? Ignore them.

I disagree that all reviewers use other equipment as a reference. Yes, a comparison, even if not stated in the review, to other equipment is probably inevitable. So what? As long as the focus is on how it measures up to the sound of the real thing.

I'll say it again: There are many "generic" qualities in the sound of the real thing, that makes concentrating on issues such as the effect of the sound of different venues, mics, mixing boards etc., while certainly not irrelevant, almost pointless. We just need to familiarize ourselves with these sonic qualities enough. Perhaps spend less time and energy on feeding the neurosis of the hobby, and more time attending live events. Not an essential pursuit in order to enjoy music and the hobby, but certainly essential if we want to bother with comparisons using any kind of meaningful standard.
It seems to me that the recording itself sets the absolute limit by which we can evaluate the reproduction capability of our equipment. As has been said before, our equipment cannot transcend the recording itself, reaching beyond to a "more true" or "real" version of the actual musical perfomance recorded by the engineer and the processed into CD, Vinyl, etc.

I'm not sure why the "Objectivist" approach to evalauting equipment would not be the prefered approach to judging a piece of equipment's ability as a reproducer of a muscial signal. The less damage (distortion?) between source and "reproduction" (electronically and acoustically) the "better".

Now it may be that there are some aspects of music reproduction that we are not able to objectively measure, and some aspects that we do not know we should measure that are relevant to sound quality (psychoacoustically). I suspect that there many measurables that relate to the accuaracy of reproduction and that are psychoacoustically relevant.

If true, all we can ask a designer to do is to engineer equipment that "measures right" (reproduces the source with minimal damage). If they do that, we can say the equipment is good to the extent is accuratly reproduces the source and passes that information to the next string on the audio chain.

With equipment that can accomplish that, our critique of the "sound" coming from our stereo might be better directed towards the recording engineers that produce sound that is accurate to the orginal perfomance.

Whether this "accurate" sound is pleasant or unpleasant is up to the listener. It would not suprise me if many people prefer sound that in some way is a distortion from the original or add artifacts that were not part of the sound of original performance in real space.
Zaikesman, consensus does have value, as do the opinions of everyone who reviews a piece of gear, as long as it is done with integrity. If you tell me the Mc Cormack amp sounds good I believe it does. Based on your comments I would tell someone else that the DNA amp you mentioned is considered to be a good amp. The issue is what have I really learned. I consider you to be a credible source for information, but ultimately what has your comment done for me?

This is certainly not to say your comments are worthless, but it does call into question the value of your comments. I’m not trying to be rude, I’m just looking at the way we think about our systems. Is your determination of ‘good enough’ going to agree with mine, or anyone else’s?

Each instrument has a specific sound. That instrument will have a specific, but different sound when it arrives at the soundboard. It will have a slightly altered sound again as it is put onto a CD or LP. Which version of that sound are we willing to accept as being good enough, or as a standard which any or all of us can accept for judging the quality of our system? Or is it even worth it to try to have a standard?

The whole issue from the outset is my own wondering whether there was a standard that everyone can use to judge quality of gear. Other than anyone’s opinion it doesn’t seem that there is a standard. I believe everyone (almost) who’s posts and ideas I am familiar with on AudiogoN when they say something sounds good. I’m sure they think it sounds good, so in their mind it does sound good. Some people are less likely to have enough credibility to have their statements accepted by me as being reliable. But none of these comments are based on an absolute standard.
Of course. It's like asking to be able to depend on some sort of standard for how good food tastes, or which people are the most physically attractive. The only difference is that in audio, we are presumably trying to *recreate something* (more so in the case of live acoustic music), which by definition implies an original creation, a standard if you will. But since we can never be certain what that original was - only that we won't be able to perfectly capture and recreate it - things become, as we see in the high end, to a large degree simply a matter of taste. Still, the concept of standards does apply, for if we stray too far from a seemingly plausible attempt at faithful recreation in the pursuit of imposing idiosyncratic qualities, the majority of us will recognize this as no longer being high in fidelity - which gets us back to the question of fidelity to what? That's where the cognitive dissonance lies - the relatively peaceful acceptance of which is something essential to achieving, if not really the suspension of disbelief, at least the blissful ignoring of it. Which is to say, Don't sweat the whole concept so much that you can't hear around it, just enjoy as best you can! Some of my favorite 'lo-fi' recordings capture primarily a feeling, not the literal sound of a performance, and maybe not even the exact feeling I would have gotten had I been there in the room with the performers. Nevertheless, I have received the artistic feeling of that document listening over a high end system and over a table radio or a car radio. The most important part of the music can easily transcend fidelity for some mysterious and wonderful reason.
Bingo, Zaikesman; the feeling "can easily transcend fidelity for some mysterious and wonderful reason". Brilliantly stated. My only qualification would be concerning the use of the word "fidelity". I would instead say: "...fidelity in the usual sense..." In other words, fidelity as usually defined by audiophiles; not necessarily the most relevant concern.

This is precisely the point. This feeling is the most important part of music making and likewise, the most important part of music reproduction. Can a component reproduce the feeling of the performance? This is in part what I mean when I refer to the "generic" sounds of live music. Although we tend to not think of the "feeling" in terms of "sound", the feeling has much to do with issues of microdynamics. The next obvious question will be: How do we know if that "feeling" was actually there at the original event, and not some sort of distortion caused by the record/playback process? Because just as with more usual concerns such as timbral accuracy and soundstage recreation, with enough exposure to the real thing, we can learn to recognize what is truth and what is a distortion.

The feeling of a good performance is a pretty powerful thing; the reason that it can transcend the LACK of fidelity of a table radio. It's not that the table radio somehow manages to reproduce the feeling because of some subtle electronic attribute, it's simply that the feeling of a good performance is difficult to destroy completely; it is that powerful. And this is what hangs up a lot of audiophiles IMO. We tend to focus on subtle differences in timbre and soundstaging, and overlook the magic. And where can we experience the most magic? At a live event.
I often think that one of the reasons I find it so easy to enjoy the music on my crappy, stock car radio is precisely *because* it eschews literal fidelity, and I know that it does, so it frees me up not to worry about such concerns and just groove. But then again, there are often times and recordings where it fails somewhat by not being able to clearly transmit content important to the musical message, this mostly being dependent on the type and complexity or subtlety (sonically speaking) of the music in question.
What a priceless 24 hours. This afernoon, I did some critical comparisons of the DNA-125 vs. my VTL's after leaving the SS amp playing all night to get it really warmed up. Then, taking a cue from Nrchy, I played a CD-R of some stuff that I recorded with a band I was in a couple of years ago. My familiarity with the sound of my own guitar and the singer who's been a friend of 20 years made for enlightening auditioning, and I have to admit that the results took the bloom off the rose somewhat as far as the SS amp is concerned. Such is the danger of first-day judgements. There are still things I think it points up that my tube amps don't do as well as could be wished for, but most of their flaws I was already well aware of. More significantly, I think I've decided upon further listening that the deviations of the SS amp are more detrimental to my musical involvement than are the deviations of the tube amps. I guess I'm back where I started, thinking even more about modding my tube amps, but an educational reaffirmation is exactly what I hoped for from this experience, so I'm not at all disappointed.

The funniest thing is though, last night I emailed the guy I bought the SS amp from, saying I may just wind up keeping it as a spare for emergencies, adding, "You never know with tube amps!" Well, no sooner had I completed this latest round of auditioning and decided I was on the right path with tubes, lo and behold one 6550 lets go with a light show! I think I caught it very quickly, so I'm anticipating nothing more serious than replacing the tube. But I was laughing ruefully...

P.S. - Update: No problems after replacing the tube. In fact, this turned out to be a blessing in disguise. When I got in there and started changing tubes and rebiasing, I began to notice some chassis reverberations making themselves heard through the speakers. Come to find that the NOS 60's Sylvania gold-pin 12AT7's input tubes I had upgraded to when I first got the amps, which I had liked for their treble extension and bass firmness, had become microphonic. Switching back to the previous GE's tidied-up the soundstage, toned-down some lower-treble brightness, and kept HF images from splashing forward - some of the areas the amp comparo had pointed up as wanting. The fun just never ends...
Zaikesman I offer my condolences on your tube. I'm glad it wasn't anything more than that!

Interesting test you ran with your new amp,and the CD-R. You are right, you know what your guitar sounds like and you know the voice of the singer. I would be interested in hearing how both of you sound with the VTL's in place. Does your Fender sound the same as it does when you listen through your vintage Marshall amps?

I think many 'audiophile's' find it easier to enjoy car audio because their expectations are low. No one thinks their car is going to sound like a concert hall, so they just enjoy the compressed lifeless music of their choice. BUT at home we demand more. Which begs the question again, what do we want our music to resemble?
I know that neither amp, nor my system generally, could exactly capture the sound I worked to achieve in the recording and mixing of the sessions. For one thing, the control room at the studio we used was purpose-built with angled walls and ceiling plus extensive acoustic treatments, and floated an image better than any home stereo I've ever heard. Hearing the mastertape during playback was so often startling in its physical embodiment of our performance that it could be downright spooky. However, I'm not depending on my memory of those sessions for my verdict on how close my system can come - I routinely took home rough and completed mixes throughout the recording process at the time, and knew then that I couldn't completely recreate what I heard in the studio when I got home.

At this late date, all I'm doing is listening to final product over the two amps and comparing for which rendition gives me more of that feeling of recognition for my own guitar and the band I practiced with every week for three years. Also, I have an innate sense of the recorded sound I was stiving for in my production, and listening over the tube amps, I am satisfied I had come fairly close to what I wanted ; heard through the SS amp, I don't feel the result was as consistent with what I was attempting to capture - it sounds more like the work of a stranger. But this still a very subjective judgement and open to question, because I've heard this disk played back on my regular system many times since it was recorded, so I'm accustomed to this particular presentation.

And I still think the McCormack is a very nice piece, particularly for it's low cost, showing amazingly little of the negative qualities I was prepared to find in an amp of its type. Having it here definitely helped me get a better grip on what's going on, and what needs to go on, with my reference amps ; I've done several other gear upgrades or substitutions over the past 18 or so months, but it had been too long a time since I heard any of it through an amp other than my VTL's. (It's no accident that it took a SS amp to show me I was having some tube-wear issues.) I can only imagine the hilarity that will ensue whenever I get around to upgrading my speakers...

Oh BTW, I was playing a Micro-Frets though a Super Reverb :-)
Nate: Actually, there were just two songs from those sessions, which were issued on two of the annual compilation disks that get released for the Xmas holidays by a local charity against hunger to raise funds. All the tracks are donated by the bands, some of whom play a live benefit as well (which we did). The stuff wasn't audiophile-type material though - we played seedy garage rock...
It would still be fun to hear it! Are the CDs all gone? If not where could a person obtain one?

I've been thought of as seedy by several people in my life.
Well, if you really want to hear humorous lyrical rewrites about Santa Claus (all songs must be holiday-themed) set to covers of tunes by The Pretty Things and The Chocolate Watch Band and raucously banged-out by a bunch of never-were's on a spree, email me and I'll be glad to send you one of the dubbed copies I so thoughtfully made up for friends and family... :-)
I enjoy reading this post from time to time. It reminds me of a saying in a song "Houses are just made of wood" It's the love of the people inside that makes it a home. An audio system is the same way. A person who loves and has a passion for music will have a different system from a person looking for near perfect sound reproduction in his sound system. One has that magic, the other is impressive. Combine both,and now your home is a palace for sure.
I just compared two systems with the same speakers, that had different tone and timbre. Out of all the many aspects of high end audio; you had better be rich if you are in pursuit of precise tone and timbre.
I disagree that unamplified music is the standard that should be used. I do agree that if listening to a system and music, one should know what the music is suppose to sound like before passing judgement. However, the only one that would know what the particular recorded music sounded like at the moment of recording. Therefore, unless you participated in the recording session, you have no idea what the music sounded like then. So, you use your best judgement as to what you like, taking into account system interaction and the room. However, my point is there are good recordings and bad recordings. I have several albums and cds that are basically great music but terrible recordings. So, they just sound bad. It isn't the system, it is the recording. Classical music is the worst violator. There are multitudes of recordings of the same piece, and they all sound different. Different venues, sound recording equipment, musicians, etc. How was it supposed to sound? only the people in the room during the recording would know.
So, my feeling is that the simple statement that unamplified music is the standard is not entirely true (to me). The standard is what the particular recording sounded like at the time it was being recording should be the standard. yes, it is very important to know what a violin, real symbol, flute, etc. really sounds like, but which flute, which violin? they all sound different and different musicians also play and sound entirely different.

So, go to a concert that is being recorded. Sit is a good venue with great acoustics, in good seats and listen to a performance, it doesn't matter if it is amplified or unamplified. The purchase that particular recording. There is your standard. You were there! You know what you heard then (or I hope you remembered), then play that recorded music back on a system. Does it sound like you remembered? Yes? then you are there. No? then there is work to do.

I know, virtually impossible to do this. So, do the best you can.

enjoy

enjoy
As I mentioned previously on another thread, a system must be able to convey Rhythm, beat and tempo. This is the ability to follow each and every musician, solo and as a whole(Thank you Ivor). System tonality, spatial perspective, dynamics, noise floor,etc. are all important. But we are listening to recordings.
"So what do we audiophiles really use as the standard by which recorded music can be judged?"

There is no standard nor need there be.

No two people like exactly the same thing.

That's like asking what's the standard for determining the taste of ice cream.

I like Chocolate. You may like strawberry. It's OK to have different individual goals and like different things (unless we are Borg of course).

Why is it that so many audiophiles have so much trouble accepting this in practice, even if they do in theory?
I just want my system to sound a closley to what the artist's intent was on the recording. I know I can't get the sound as it was being played in the live arena or studio. I just would like the music to sound as close as possible to the music that's on the actual CD, LP or whatever medium it's on.