@yowser Your goal here is, I assume, to improve performance.
To that end, by bypassing the power supply, what you want is uniform output impedance of the power supply at all frequencies. So the correct dielectric may not be the one that is also the most expensive!
I'd stay away from oil and paper parts as they have a way of developing electrical leakage and may also leak physically. We stored a bag of such parts for about 10 years and upon retrieval, found that the bag smelled of oil. So for reliability I would not use such parts if DC voltages are involved.