would I be better off...

I agree with Bomb. I would keep your amp and get an outboard DAC.
Please tell us more about your setup: computer, file types, playback software, speakers.

I found the DL III to be a fine DAC and very worth the used price i see then go for here on AG. Yes the USB was included for convenience. I used a USB/SPDIF converter as well. Problem is that it only has a single coax input so makes it difficult to use a transport and computer via coax. I sometimes just like to play a CD, so I went for a DAC that had more inputs. I do like to use an external USB converter as i believe there are many high quality DACs available where the USB input is the weak link. That said, given your budget the suggestions by Realremo are sensible. I will add that I put together a Music Fidelity V-DAC and V-Link combo buying used here and E-bay for ~$160. There is a V-link for sale now here for $75. It would work well for CD quality files and streaming from Spotify. Would also allow flexibility in purchase of a used DAC.

I am not a PC user, got into the MAcs. I rip my CDs as AIFF which is equivalent to WAV for PC. I do know that for PC, JRiver is highly recommended.

There is a Micro Mega MyDAC for sale here, asking price is $250. I think it would be worth trying out. I have not heard one, however I mention this option because this DAC has gotten great reviews by audio press and owners as being very in the under $500 price class. If it did not work out as you liked it could be sold without much loss.

Great! Should be a good addition to your Yamaha.