I understand your reaction to bypassing the passive. It appears you are not substituting anything new but rather just eliminating one link, namely, the passive preamp. Every signal carrying device has limitations, even if it is just the RCA connecting jacks, input selector switches, and a simple attenuator, and if you can eliminate them chances are there will be a sonic improvement. Let's also not forget you eliminated one interconnect cable as well.
In my case, I still find a very high quality active preamp best suits my system and sonic priorities. But I use it with a single set of hardwired RCA input jacks, i.e. no selector switch, and there was a clear improvement in sound quality when I did that.
In my case, I still find a very high quality active preamp best suits my system and sonic priorities. But I use it with a single set of hardwired RCA input jacks, i.e. no selector switch, and there was a clear improvement in sound quality when I did that.