I think you should stick with the Yamaha for now and for quite a while. Amplifiers should not make much of a sonic difference, and by and large they don't. AV receivers are not ideal, but a recent one from a decent brand like Yamaha should not be a serious issue, and the power output should be enough.
The speakers and their interaction with the room are what really makes the sound of a system, and good speakers are expensive. I am not sure what your budget is, but you may have to spend more than you thought you needed. In a room like yours, how about the Harbeth SHL5+ on their own or their M30.1 augmented with a pair of small subwoofers? Both options are expensive but worth every penny. They are also an easy load, so your Yamaha should be fine with them. As an AV receiver the Yamaha will also have the advantage of room eq of one kind or another so you will not need to spend money on that. If good speakers are beyond your current budget, just wait and save. You don't have a bad system right now.
The speakers and their interaction with the room are what really makes the sound of a system, and good speakers are expensive. I am not sure what your budget is, but you may have to spend more than you thought you needed. In a room like yours, how about the Harbeth SHL5+ on their own or their M30.1 augmented with a pair of small subwoofers? Both options are expensive but worth every penny. They are also an easy load, so your Yamaha should be fine with them. As an AV receiver the Yamaha will also have the advantage of room eq of one kind or another so you will not need to spend money on that. If good speakers are beyond your current budget, just wait and save. You don't have a bad system right now.