What's best bookshelf speaker for an 8x10 room?

The Vandersteen Model 3 would be my first choice.

Several will work well in spite of what some will tell you.
Spendor and Harbeth are the two most popular choices.
Linn monitors work in small rooms.
I have the same size room, right now using Kudos Super 10.
But also have a pair of these:
http://guruaudio.com/
I would recommend them

Without knowing what amplifier you have, it would be impossible to recommend a speaker; even if was being used in a small room. Besides, speakers are so subjective, what sounds good to one person, may not sound good to you. If any component needs to be auditioned, its certainly speakers.

Also, you indicated you are going to use them on a bookshelf. Therefore, you need to look at front ported speakers only. If you press rear ports against a wall, you won't be getting all the bass the designers intended. For example the Kef LS50's that were suggested, are rear ported and those ports should be 18"-36" away from a rear wall. And I have never seen a shelf that deep.

So my suggestions is to know your amps capability, makes sure it matches the impedance and sensitivity ratings of what speakers you are auditioning and consider front ported models only. A far more intelligent approach to find the speakers that are best for you. Good luck and rock on brother!

BTW
my recommendation above is for the Guru Juniors only. They are meant to be placed right against the rear wall, or in a bookshelf

I can testify that the Merlin TSM will do well near the front wall. In a second system, I have a pair placed 10 inches into a huge room (16x25), resting on their sides in a large, open-sided wall unit, driven by a Manley Stingray II with a Marantz SA-11s2 SACD source and a small, sealed Von Schweikert sub. In your room the sub would likely be unnecessary. Naturally, they'd image better on stands four feet into the room but they do quite well in this compromised arrangement.

Paraneer gives good advice about avoiding rear ports so near the front wall. The TSM has a sealed cabinet, as you probably know. I have a pair of front-ported Omega Grande 8R's, which also work well there, but the Merlins are quite a bit better.

For a room that small, ProAc 1sc's should work very well.

I've heard System Audio Mantra 10 speakers are very special and very musical. You can give Oz a call at Let There Be Sound and he can tell you more about them. I can give you his info if interested (or you can google the store and tell him David referred you). He has been very helpful to me with all my musical gear. The Mantra 10 retail for $1500. Check out the System Audio website. The Aura series is still good but the Mantra are even better. It is a great value from what I've heard.

Sennheiser HD800s.

Nice pair of ATC SCM 19s for sale here now. If rock/hard rock is your genre these will knock you off your chair. In a good way. Read the reviews, but ATCs excel in dynamics, punch, timing, and outright midrange clarity and force. Everything you need if hard rock is your cuppa. A little over your budget, but given what you're looking for they're more than worth the stretch IMHO. Best of luck.

I would not go for a very small speaker in a small room. At low volumes it will sound lean. I use Harbeth C7 and B&W 602 which is a better size for my small (10x12) room than my Proac Tablettes, which I use in my bigger room.

Thanks guys! I now have lots of research to do.

North Creek is another vendor, like Guru, who voices speakers to work close to boundaries, often a very useful attribute in small spaces. They are very good.

Salk Wow 1 or Neat Iota.

Echo North Creek kitty Kat Revelators. Always under the radar but superb speakers designed to play again back wall. Can't vault them. My best buy in speakers for a long time