The streamer and server, and even the network cables and switches all do matter. Will upgrading them make a considerable improvement to everyone? No. Why? It depends on the remainder of the chain and the individual’s listening ability.
Equate the digital front end to a vinyl rig. Would one put a $5,000 moving coil cartridge on a $300 turntable and built-in phono stage? Likely not, as the delicate signal carried by the cartridge would be destroyed by resonances in the tonearm, rumble in the table, and poor tracking, all which introduces noise. And would one do so if the speakers and cables could not articulate the improvement? Doubtful.
One poster suggested upgrading the streamer first and get a better DAC down the line. I would suggest otherwise, as the investment into a streamer / server will not be fully realized until you have a DAC that can truly support its performance.
Last year, I was at this crossroads. I wanted a more natural sound to my digital front end. I had a PS Audio DirectStream DAC fed by a NAD M50.2 dedicated streamer (former Bluesound graduate here). I went on a big exploratory with Lampizator DACs, Mola Mola, etc. What ended up making the bigger difference? Replacing the NAD M50.2 with an Aurender N30SA. It almost made me give up vinyl. Turns out the DAC was able to produce the fidelity of the signal, and it was the source that was lacking. The streamer made a considerably bigger difference in sound than upgrading the DAC. I have since also moved onto a different DAC which equally improved the sound in as big of a jump, but it required an even larger investment.
A few weeks ago, I demoed the difference between a PC playing Tidal through a good USB cable vs the Aurender playing Tidal with a USB cable. Both of the guys that were sitting for the demo dropped their jaws like someone slapped them in the face. The tonal balance, textural richness, and lack of fatigue in the Aurender is just astounding. Bits are not just bits.
The Innuos is a good move and should resell more easily when you are ready to upgrade. Your 205 is capable of taking advantage of it, but if you much higher in the chain, I would also suggest you prepare yourself to also upgrade your DAC to reap the benefits.