I had a similar experience when conducting ABX testing with amps (with all other pieces of electronics, I was able to select the proper item -cables and components - far better than 50% of the time). The only component that was not discernible in level matched ABX was the amp, which was shocking to me! The most bizarre aspect was that it was not possible to score well in ABX even when the amps being compared were SS vs. tubed!
These results were written up by myself for Dagogo.com, as I reviewed the Audio by Van Alstine ABX Comparator. It was a fascinating experience, and I'm glad I took the time to explore it. Van Alstine agreed with my results, even with the aspect of the difficulty of distinguishing between amps in ABX.
However, and this is critical, the similarity between amps was only when conducting ABX. Apart from that particular condition, or when reverting to normal usage in an audio system, the differences reasserted themselves. Therefore, I concluded that: 1. ABX is not necessary to establish differences between gear, 2. It does not represent real world conditions of listening, and has quite limited applicability to it, and finally, 3. That we have phenomenal ability/sensitivity to perceive minute gradations sonically, but poor short term acoustic memory. It took a LOT of concentration in a custom room with very low noise floor to conduct the ABX testing with accuracy.
Consequently, after conducting my own exploration, I am content to spend less time thinking about ABX, and more time building better audio systems through very active, hands on methods.
Finally, I have done comparisons as well with DACs that were said to not yield differences of upstream components due to them being "bit perfect", etc. However, I did not find that to be the case. Whether they would be similar to amps, or not, in ABX testing, I do not know. :)
These results were written up by myself for Dagogo.com, as I reviewed the Audio by Van Alstine ABX Comparator. It was a fascinating experience, and I'm glad I took the time to explore it. Van Alstine agreed with my results, even with the aspect of the difficulty of distinguishing between amps in ABX.
However, and this is critical, the similarity between amps was only when conducting ABX. Apart from that particular condition, or when reverting to normal usage in an audio system, the differences reasserted themselves. Therefore, I concluded that: 1. ABX is not necessary to establish differences between gear, 2. It does not represent real world conditions of listening, and has quite limited applicability to it, and finally, 3. That we have phenomenal ability/sensitivity to perceive minute gradations sonically, but poor short term acoustic memory. It took a LOT of concentration in a custom room with very low noise floor to conduct the ABX testing with accuracy.
Consequently, after conducting my own exploration, I am content to spend less time thinking about ABX, and more time building better audio systems through very active, hands on methods.
Finally, I have done comparisons as well with DACs that were said to not yield differences of upstream components due to them being "bit perfect", etc. However, I did not find that to be the case. Whether they would be similar to amps, or not, in ABX testing, I do not know. :)