I hope Steve Nugent of Empirical Audio reads this so he can give you an expert answer.
As far as my own two ears are concerned, I feel I can put your "fundamental doubts" at ease. Even with a modest USB to SPDIF converter like the TRENDS, the sound holds its own with very much more expensive CDT/DAC combinations and using Steve's SPOILER with PACE CAR reclocker you get a practically jitter free signal, the sound of which easily surpasses both the Zanden Combo as well as the DCS stack in my rig in all the important parameters we audiophiles love.
As far as my own two ears are concerned, I feel I can put your "fundamental doubts" at ease. Even with a modest USB to SPDIF converter like the TRENDS, the sound holds its own with very much more expensive CDT/DAC combinations and using Steve's SPOILER with PACE CAR reclocker you get a practically jitter free signal, the sound of which easily surpasses both the Zanden Combo as well as the DCS stack in my rig in all the important parameters we audiophiles love.