Yes, too much can be an issue, definitely. I keep mentioning them, but that’s OK because they are timeless classics: the Epi 100 are 21" tall and about 10x10 side and back. They are big enough to sound beautiful and full, and small enough to be super versatile and practical in real life.
Also, a big consideration for speakers is How do they sound off-axis, outside the sweet-spot...? If they’re only great in one particular seating position, then they’re not great for life. So as you audition, stand off to the side and see what happens. Smaller speakers can be remarkably great for off-axis, as their cabinetry can get out of the way. @audioman58 +1 for that