You'd be hard-pressed to make a "bad" purchase if you stick with a Hewlett-Packard or Creative Labs model, amongst others. They'll work great, last a long time, and are easily obtainable at Best Buy or CompUSA.
There is a lot of music available on the net, but you'd have to first define what "high quality" means to you as well as your morals on the issue of "available music" to give you a definitive answer on how much is available. There is a lot more 128Kb MP3 music available than true CD-quality or even 320Kb MP3 music. Nobody is legally giving away CD-quality music over the net for free, at least of bands that are at all famous.