If you plan on having only one source, start there. If you plan on having several, do the main IC.
either way, it's just a starting point... not a finishing point, right? And you gotta have something somewhere. Well Hell, everywhere.
My own plan is like this.... nice spkr wires, not cheap nothings, so you can hear the changes being made upstream.
Decent IC on main, and source.
Power cables.
Go back to sources or main IC and upgrade there.
Now finally upgrade the spkr wires.
Buy new components and begin again as and where necessary.