I have two modified PC's that do the streaming on my two listening systems, both using USB connections to my DAC's. I agree that a bone stock, unmodified Windows based PC should not be used for streaming without modifications. Windows is horrible at processing audio and was never intended to do that without modifications.
I also agree that the PC should only be used for streaming and not doing CAD drawings, have 24 Excel spreadsheets open, doing your YouTube video editing, etc. Hence why our OP is having success with the process of slimming down what is running.
I have found that even the free version of something like Fidelizer will make significant improvements to the USB audio quality on an out of the box desktop or laptop PC mainly by modifying how Windows processes audio.
My experience has been that mods such as audiophile network and USB PCIe cards (NOT powered by the mother board), linear power supplies, stripped down Windows 10 Pro and higher quality internal components (power supply, motherboard, ECC RAM, etc.) all add up to making the PC much more of a robust, quieter and better sounding streaming device.
It's my understanding (as @mlsstl mlsstl points out) that streamers are essentially computers, but all the hard work of modifying a computer has been done for the end user by optimizing all components and software for the sole purpose of streaming music (not processing CAD files, Excel work or rendering video). Hence their widespread recommendation over using a PC.
But I think if one is well versed in how a PC works and can successfully modify the PC, it can be a good source of USB digital audio. I have had a bunch of fun on my journey modding PC's to produce better sound. Each small step I took produced better sounding audio.