You’re on the right track. As long as the digital data is getting to your DAC intact, there’s no reason to get a ’better’ streamer. It’s already doing the job it needs to do.
Jitter isn’t much of an issue. For example, most USB interfaces simply drop data packets if the data stream has too much jitter / error and this will manifest itself as an interruption in playback. If you’re using another digital interface like coax SPDIF or optical, large amounts of jitter will contribute distortion to the signal. If you want to hear what that actually sounds like, here’s a link - Let’s listen to some jitter simulations. Similarly, here’s what an underperforming digital interface sounds like (e.g. not enough jitter / error to cause a dropout).
Noise is another supposed bugaboo. Guess what? If you don’t hear any noise when your system is turned on and set to the volume you listen to, then it’s not a problem.
Digital signals are actually very robust. Think about it this way - if digital signals were fragile, how would the astronauts in the ISS be able to get a 600 Mb connection? How would people be able to stream audio or video from a service that stores the files in servers (which incidentally were probably also copied over a digital network) that are hundreds of miles away?