It's not as simple as a "loudness value" nor is it a relative volume level comparison between frequencies.
A speaker attempts to reproduce what is on the recording as accuarately as possible. Since we can hear in the range of 20 to 20kHz, most speaker specs cite this range. If the speaker was dead accurate, the "plus/minus dB" spec would be zero. What that means is that all frequencies would be reproduced AS RECORDED, not that all frequencies are the same volume or have no relative loudness between them.
But speakers are not accurate transducers. Some frequencies will not play at the same level as the recording; so they deviate from an "ideal". This ideal level is 0 db reference. So in your case, the +/- 3 db from 50 to 20kHz means that frequencies vary from the ideal by no greater than 3 db. But down to 40 hz, some frequencies vary by 6 db. Just a way of saying that the speaker plays down to 40hz but you may not hear much. In fact speakers play down to well below 20 Hz, it's just that you cannot hear it. The dB value at, say, 15 hZ could be - 18 dB.
The other thing to note is that not all frequencies vary by the +/- db value. One, a hundred or a thousand frequencies can vary. Some vary by that value at both ends. For example, at 200 hz it can drop by (-3 dB) and at 500 Hz can rise by + 3dB. Within spec, but a hell of a 6 dB suckout at the midrange (3 db on each side equals 6 dB). Whereas if only one frequency varied by 6 dB in a 20 to 20 kHz (+/-6dB) range, all the rest of the frequencies would be zero variance and you'd have one of the best speakers in existence. Even though it's +/- 6 dB, it a lot better that one with a +/- 0.5 dB.
Specs are misleading with speakers. The above is one reason why.