Typically passive preamps do not exhibit this trait, but depending on the rest of your system you may have trouble finding what you are looking for. A good passive preamp should not add or subtract anything from the equation. However, impedance mismatches and cable issues could cause the problems you are describing.
In my experience I have preferred transformer or autoformer based passive preamps versus their resistive counterparts. The TVC/AVC passives tend to match up better in most systems. Impedance matching is important with passives. Just changing the setting on the volume control could impact the output impedance on the passive. Also, since we don't know the rest of your system there are a few things you should look for moving forward.
1. Your source has enough output to drive the passive (2V or better with low output impedance).
2. Your amp should have high input impedance (50k or better) and be sensitive (1V or less to drive it to full power).
3. Try to keep cable lengths short between the source and passive and passive and amp. This is more applicable to the resistive passives, but probably a good practice with any type.
If it were me, after determining if my system was passive friendly I'd try a TVC/AVC and see what happens. Lots out there to try that you can find used for good prices. Promitheus, Antique Sound Labs, and Sonic Euphoria come to mind.
Also try reading this:
http://www.stevens-billington.co.uk/page102.htm
In my experience I have preferred transformer or autoformer based passive preamps versus their resistive counterparts. The TVC/AVC passives tend to match up better in most systems. Impedance matching is important with passives. Just changing the setting on the volume control could impact the output impedance on the passive. Also, since we don't know the rest of your system there are a few things you should look for moving forward.
1. Your source has enough output to drive the passive (2V or better with low output impedance).
2. Your amp should have high input impedance (50k or better) and be sensitive (1V or less to drive it to full power).
3. Try to keep cable lengths short between the source and passive and passive and amp. This is more applicable to the resistive passives, but probably a good practice with any type.
If it were me, after determining if my system was passive friendly I'd try a TVC/AVC and see what happens. Lots out there to try that you can find used for good prices. Promitheus, Antique Sound Labs, and Sonic Euphoria come to mind.
Also try reading this:
http://www.stevens-billington.co.uk/page102.htm