Go to http://www.silentpcreview.com and you'll find a recommended list of quiet power supply units for your needs. A power supply provides DC power to your PC's components, so it definitely can affect your audio if your current power supply is severely overloaded (i.e., you've added some more disc drives, a high performance AGP video graphics card, or some other electron-hogging devices). For PC audio, you should want a quiet power supply unit as this component is usually the noisiest part of a PC system with its cooling fan running.
I don't know what kind of audio interface you have, but if it's a consumer card like a SoundBlaster, SoundBlaster Audigy, Turtle Beach or some other low cost sound card, you can make a big difference upgrading to something like an M-Audio Revolution, M-Audio Audiophile 24/96, Echo Mia MIDI or even a Digital Audio Labs CardDeluxe. These are all PCI-bus, internal sound cards and they perform very well within a PC case enclosure. Most USB audio interfaces really don't perform any better than PCI bus sound cards, because the USB 1.1 bus doesn't have the bandwidth, it isn't that stable and it is actually more prone to system conflicts than the PCI bus. USB 2.0 was a huge improvement, but I am not aware of many USB 2.0 audio interfaces on the market. If you consider an outboard audio interface, look into a Firewire audio interface. Firewire is more optimized for large, uninterrupted data transfers. Firewire was developed for handling electronic media data, USB 1.1 wasn't.
I don't know what kind of audio interface you have, but if it's a consumer card like a SoundBlaster, SoundBlaster Audigy, Turtle Beach or some other low cost sound card, you can make a big difference upgrading to something like an M-Audio Revolution, M-Audio Audiophile 24/96, Echo Mia MIDI or even a Digital Audio Labs CardDeluxe. These are all PCI-bus, internal sound cards and they perform very well within a PC case enclosure. Most USB audio interfaces really don't perform any better than PCI bus sound cards, because the USB 1.1 bus doesn't have the bandwidth, it isn't that stable and it is actually more prone to system conflicts than the PCI bus. USB 2.0 was a huge improvement, but I am not aware of many USB 2.0 audio interfaces on the market. If you consider an outboard audio interface, look into a Firewire audio interface. Firewire is more optimized for large, uninterrupted data transfers. Firewire was developed for handling electronic media data, USB 1.1 wasn't.