.3 mV is only 60% of the .5 mV minimum. That can be significant.
You might have to turn the volume up further than normal at best to get to normal listening volumes, or the volume might not ever get as loud as you like. If you have to turn the volume up close to max, you might get more noticeable background noise levels also.
If you do not listen at higher volumes, there is a better chance of it working OK. Try it and see. If it doesn't, then just be prepared to add a good step up device. This will probably cost at least a few hundred as well for a decent one so as to not degrade the sound, so be aware.