Most amplifiers reduce distortion as volume goes up. Take a look at the distortion vs. output of a linear amp (figure 5) below.
Is it the job of a preamp to be sure the amp sounds the same at all levels or is this just impossible to make the sound that linear?
It's main job is to act as a buffer with variable gain. The buffer part is that the source doesn't see a variable resistance. Tubes often lack an output buffer though so the preamp itself may be subject to volume control variances. The preamp can't compensate for problems in the amp though.
The difference you are hearing may be due to speaker impedance / amp interaction. It's possible one amp is responding more to the speaker load than the other. If that depresses the midrange, at low volumes that may be helpful.
https://www.stereophile.com/content/luxman-l-509x-integrated-amplifier-measurements