Digital is all about managing the noise which is something you don't really hear until it is gone. Its what makes digital thin and harsh and edgy. When its managed the sound will swell with richness and texture and detail in a black background as you increase the volume.
Upgrading gear will give incremental improvements but is really a band aid on the underlying problem.
@steakster adeptly listed areas to address before gear changes.
I will suggest also adding at least one purpose built network switch (my experience is 2 are better) and would suggest an EtherRegen if you are only using one and dedicate it for your streamer as it uses an optical break to remove noise from reaching your streamer.
I will also suggest a product like the Akiko Corelli and or one of the CAD grounding products (I have 3 Corelli's and 2 CAD units. Its tempting to believe they are hocus pocus because who wants to spend that much on something that your not sure isn't snake oil.
Really look into a dedicated AC line for your analog and one for your digital. It is surprisingly affordable in most cases depending on the drywall repair needed. I cant understate the effect on digital this has.
And never have a SMPS or LED lights on the same line as any digital equipment. A dedicated line will remedy this.
A filter on your network cable into the streamer will help. Look at Network Acoustics or SOtM.
Try to avoid USB, use Coax, BNC or AES or Ethernet with a good cable. There are many DDC/reclockers that can be used to convert the USB if you lack the outputs from your streamer.