Are passive preamps better?

Why not try a pair of Endler or EVS attenuators. These are stepped attenuators that include custom RCA jacks. One end is male and plugs into the RCA inputs on your amp. The other end is female and is where you will connect the output cables of your source. This should give you a good idea of how a resistor based passive device will sound and they are relatively inexpensive to try. I believe one or both brands may come with a trial period, so you can return them if you don't like them.

I personally prefer transformer volume controls as for the most part they avoid the impedance matching issues of resistor based passives. I've been using a custom built unit using S&B transformers for a while now with excellent results.

The Endlers aren't the last word in a passive attenuator, I preferred the EVS, but my reason for using them was to attenuate the gain from an active preamp/amp mismatch. In your case, to evaluate whether or not passive is an option for you, the Endlers will give you a glimpse, but eventually you'll need to move up to something like a Gold Point, Placette, Reference Line, Prometheus, Django, or Sonic Euphoria to hear the real value of a passive.

The Bent Audio TAP, soon to be back in production can be ordered with remote. The old version used transformers (S&B) while the new version will use autoformers (Intact Audio).

Bent also makes a resistor based passive.