I do agree with this in that my wife and daughters are hardly hifi people or audiophiles but they have commented when something is changed in my system for the better or worse, and this just from being around and hearing the music from afar. What they say is usually pretty spot on.
WAF - music to my ears
My wife isn't much into my audio equipment and her most common request is for me to run it down. I genuinely do not listen to it loud very often and almost never when she's around, but I'm conditioned to push it up a couple extra notches so that I can comply with the turn it down requests and still have a reasonable listening level.
The other night we were sitting on the couch and as I was scrolling through the list of albums I came across a Holst Planets that I know she enjoys. We started with Mars thinking it was her favorite Jupiter, so after finishing the song we moved on to her favorite. It was music to my ears when she noted that it was a bad recording. I promptly did a quick search for the other version that I have (that was purchased for this very reason) and we enjoyed the entire album together after the kids went to bed.
She doesn't seem to notice any of the changes/improvements that I do to my system, but at least she can recognize a poorly recorded album.