A new Android smartphone or one with an app could easily support all of those formats. Maybe a USB C dongle with a headphone out...although those are rarely implemented well and lack in terms of sound quality and output power...so look carefully.
Contrary to what some audiophiles may tell you, converting an .mp3 to .wav doesn’t make it a .wav !!! I can’t believe how many times I’ve had to explain that lossy files can never become lossless files. They are missing information...inherently defective.
My suggestion is to stick to .wav files. CD quality is all you need to enjoy the full extent of music. For $50, perhaps a used Sony Walkman digital audio player would work.