Room Acoustics

I have a question concerning acoustic panels. I’m currently trying to determine what I need and how to set it up in my room. GIk advised that I put as many bass traps in the room as possible, strategically positioned in the room and, in the first reflection points. Secondary reflection points could either be covered by a combo of diffusion/absorption or diffusers. One question I have is, is it necessary to have so many acoustic panels both traps and diffusers, if you don’t play music very loud? Do acoustics matter most when the volume is turned up and the walls rattle? Appreciate some advice. Thanks.