It comes from experience. I have never experienced anything in audio that could be 1) Verified , and 2) Not measured. Usually #1 eliminates almost everything, and when it does not, #2 takes care of the rest. Every single time where I have heard, or experienced a verifiable sonic difference, there was also a measurable difference. Not just measurable, but above what are accepted limits to hearing sensitivity/discrimination, and significantly different.
I am not talking single point, THD either, but full scans, i.e. THD across frequency and output power, IMD with 2-3 points, and multi-point at multiple powers. Then again, just taking two time correlated streams (digital source only of course) can be highly revealing.
I think we both have respect for isolation, at least for the analog. I do believe a lot of the claims of "noise, RF, pick your poison" getting into the DAC clocking is for the most part suspect. If you know RF, you know that there is more RF coming off that trace 1/2" away then you are going to pick up externally. If the people who made those claims knew depth of D/A design, they would be far more worried about the D/A analog reference :-) ... but also not difficult to regulate for audio frequencies and eliminate noise.
I have convinced myself of things I heard that I later proved were not there. You are so hyper focused that something must be there that you convince yourself there is.