Stuck: 20-bit/96kHz or 24-bit/96kHz DAC?

I'm a new audiophile so I don't pretend to know all the issues with regard to formats, but here's the rub:

I want to get the best sounding *used* CD player I can afford. Some years ago, the CAL CL-20 was a highly reviewed CD player, and it has my interest. I'd like to compare it's sound to a Toshiba SD-9200. I've heard the latter, and forgetting all about DVD-A and the player's usability faults, it makes an ordinary CD sound marvelous.

The CAL unit, like other older units, has a 20/96 DAC. The DAC in the Toshiba plays up to 24/192 for DVD-A. Assuming for the moment that the choice is between 20/96 (the CAL) and 24/96 (whatever else), how much sound performance am I giving up if I go with the 20/96?

I suppose what is really bothering me is something like the frustration I feel when my computer components become dated. Sure, they worked fine in the day, but now there's better to be had. I haven't yet auditioned 20/96 and likely won't be able to A/B test with a 24/96. The main point: if I go with a 20/96, am I buying something that I'll toss in a few years for playing red book CD audio?