What actually determines volume power? Is it watts?

As others have mentioned, the speakers will be the biggest determinate since it represents the load on the amplifier.

But to answer your question directly:

Amperes = Watts/Voltage  or to flip this around Watts = Voltage * Amperes.  There's more to it than this, but I won't go into more details because the calculation of power to watts is pretty straight forward, but unhelpful to answer your question. Plus most manufacturers won't give you enough information to answer this question.

If you like your speakers, you either need to buy a much more powerful amplifier or if your amp has a subwoofer out, you might be able to get more perceived loudness by with a powered subwoofer, depending on the kind of music you listen to.

The Yamaha is an integrated amp. So the volume control follows the preamp section gain structure. If that unit allows the use of an external preamp with more gain, the volume knob won't have to be turned up as much, and the power amp section will respond to the gain of the preamp. 

My tube preamp has much more gain than my SS. The volume heard when the tube preamp is at 11 o'clock is equal to the SS volume at 1 o'clock.

So if possible, try an external preamp with more gain and you won't have to turn the volume control up as much. 

The volume of output power is best measured in the amperage output.  Output in wats is too easy to manipulate by the manufacturer. 

Why not borrow an other amp from a friend and see what that does in your system?

We need to define a couple of things. 

What is your top listing SPL?

What are your speakers, what is the sensitivity, power handling?

Music source? 

Think you might need a whole new system to keep you happy. My amp is 200w high current, but I have 100db sensitivity on my speakers, they say they top out at 116db, On my setup, you cannot be in the room at 70% volume. It's uncomfortable. Most of my listening is around 1w of power, but at my listening spot is 70ish DB. 

Another approach, assuming you don’t know anyone that would loan you an amp to try out is to buy an amp from a company that allows a trial period and a no fault return policy if it doesn’t meet the need. VTV produces a range of class D amps that have a trial period as part of the terms of sale and it might be worth exploring that avenue. If you have an aversion to class D as some people do, this would not be a choice you would make, but I would encourage you to consider class D as a possible solution. Currently produced modules from Hypex and Purifi have excellent sonic characteristics along with the advantages of light weight, high power, especially for low impedance loads. Plus they are extremely efficient so power consumption is considerably less than any other possible choice. That might not mean much to you if you are running a 2 channel only system. But electric power costs are going up and quite substantially in my area. I think class D is the future of high end audio, especially for home theater, because of all the advantages it brings to the table.