Hello OP
As others have posted, you listening room may be your constraint. My audiophile nephew told me in the past that he has gained the most impact by improving room acoustics. At first, I did not really dig that, until no matter what I do, the changes are just incremental. One day, I contacted ATS acoustics and spoke with their support. I told them my room and my gear. they suggest the panels and I just got everything they told me (see my profile)
Once all installed and after a few days, I notice that I can bring up the volume and I can hear more into the music. I am a bit deft, but I am sensitive to tone, timbre and spatial differences.
Since you have the PAP speakers, which i have heard, you are good to go. BUT!!! That speaker needs to integrate with the room; it is so ROOM dependent. The better the room acoustics, The better that sounds. IMHO, you will be best served with Room Acoustic treatments.
Lastly you mentioned about Prima Luna upgrades (Caps and resistors). To me that is another topic all together. I am very familiar with this as my 845 Audio Note clone power amp was built with all premium parts such as Mundorf, Audio note and Dale resistors and Japan Irons. It was assembled that way. When we did a shootout between an Audio Research and VTL amp, my clone easily equaled the VTL and surpassed the AR.
Hope you find which path you want to take
Again, this was in IMHO only.