Not quite the same comparison, but there was a period of 12 months about two years ago where I was strictly using headphones.
I used the HPA-1 as my headphone amplifier. When I transitioned back to a proper 2 channel rig, I kept the HPA-1 for a while as my preamp.
While it did perform admirably, it wasn’t nearly as good as the Pass XP-12 I demoed, and even less so compared to the XP-22 I ended up purchasing.
The HPA-1 is a great headphone amplifier, and a solid preamp. It has limited functionality as a preamp however, having no remote control option, two inputs, and unbalanced.
If you aren’t focusing too much on headphones, I would go with one of the dedicated Pass preamplifiers.