Based on my testing, my upsampling audio from 16/44.1 kHz to 24/192 kHz in Audirvana (also tried DSD) DID not produce a noticeable difference for me either. This is probably because upsampling doesn’t add new information; it interpolates data points based on the original signal. The perceived improvement depends on your DAC, playback system, and personal hearing sensitivity.
Some DACs handle higher sample rates better, but others may perform optimally with native rates. Upsampling to integer multiples (e.g., 44.1 to 176.4 kHz) is often recommended for better results.
Ultimately, I suggest trust your ears and experiment with settings to find what works best for you. It is the synergy between all components that make a system sound good and especially the room.