IME everything in a digital source chain right up to the DAC is important, cables too. Also the differences between an N20 and A8 will become more appreciable the more high-end all the rest of the system components are.
That being said I think for most folks in most systems an A8 is all that is needed, a great product at an even better price.
I have a Lumin P1/L2 combo and the fiber optic connection has been a big sonic upgrade, so back to cabling, major sonic upgrades can be had without buying new components.