Would your familiarity with the music played effect your test performance?

I agree about ECM - for decades they've maintained a consistent, very high level of sonic quality. At one time years ago, I would just buy any vinyl releases on the label I saw for sale regardless of my familiarity with the artist and was rarely disappointed. And your comment on "knowing" their sound palette well enough to make comparisons speaks well of them.

With regards to most studio rock recordings, I would say, for the most part yes. Mostly because of familiarity.

But with classical and acoustic jazz, not so much.

I am pretty much only a fan of modern, avant-garde and contemporary classical music (the atonal, dissonant, ’thorny’ sounding stuff).

But since, for the most part, all classical recordings are recorded in such a way, as to capture the natural acoustic properties of the instruments, ambient and spatial information of the venue and the musicians within the venue, I could very easily listen to some 19th or 18h century classical music, which I don’t care for, but still get a positive judgement of the ability of the system to reproduce dynamics, timbre, soundstage and image.

In other words, the recording quality of the LA Phil, recorded at Disney Hall, playing some old Beethoven or Brahms piece, which would bore me, would still sound as well recorded as a contemporary piece by Unsuk Chin, Thomas Ades, Joan Tower, etc, which I would enjoy musically. But I could use either for evaluating audio gear.

Much the same with acoustic jazz. I could grab almost any brand new ECM label recording, for example, even if I am completely unfamiliar with it, but still know the high quality of reproduction to expect. And therefore, still be able to use it for evaluating gear.

Totally agree, though there is also a lot of newer music that I'm just as familiar with. But yeah, familiarity with the music will give you clues as to what to listen for....  

I view listening music for testing as similar to going on a trip with a map.  You know what's there in the music.  How high the high gets, how low the low goes and musical and 3-D it "can" be, when listened to on a really nice system.  If something becomes overpowering or isn't there.  Or, if the music sounds constricted, sharp etc.. 

Absolutely, I love classical music and can't get my head around most of the other genres , therefor I can hear much more nuances between the instruments and the airspace around them  making testing far easier.