paranoid listener-damaging speakers?

Without a meter, you're totally guessing. With a dB meter, you can loosely define your margin of safety.

You get the specs for your speakers' sensitivity (e.g., 87 dB @1 meter w/1 watt input), and their power handling capacity. These specs may be a little harder to get in your case since your speakers are custom made from a (presumably) Eton kit. You get the specs for your amplifier (40 wpc).

You double the amp power output for every 3 dB increase in SPL. So if, for example, your speakers are putting out 87 dB at 1w input, they're theoretically putting out about 16 dB more (103 dB) at 40 watts input. You may be able to add 2-3 dB output to the 1w input because you're listening to 2 speakers instead of one and another couple dB because you're listening in a room instead of an anechoic chamber.

Anyway, if you know the sensitivity rating and the max power handling, you can measure with a Radio Shack dB Meter when your speakers reach an SPL approx. equal to the amp's clean output max, after which it starts to clip and could damage your tweeters. Or if your speakers can't absorb as much power as the amp can put out cleanly (unlikely in the case of 40 wpc), you turn down the volume at the point the speakers are putting out more SPLs than their power handling indicates they can absorb.

For example, if your speakers have a sensitivity of 87 dB you hear distortion around 103-106 dB consistently from recording to recording, it's probably your amp being overdriven.

On a small two way when it sounds really "loud" then there is usually distortion either on the system or the recording. It is not well known but undistorted music can be played much louder without fatigue or sounding loud than distorted music many modern pop/rock CD's are badly distorted from audio compression -they sound harsh and loud at modest SPLs.

I'M NO EXPERT. BUT IF YOU HEAR DISTORTION, TURN THE VOLUME DOWN. IT'S ALWAYS BETTER TO FIND THE SOURCE AND MAKE ANY NECESSARY CORRECTIONS BEFORE PERMENANT DAMAGE IS DONE. IMHO- DISTORTION IS LIKE A TOOTH ACHE. ADD YOUR OWN CLICHE'S HERE...

I would worry more about damaging your ears. Eton drivers are only $200 each or so. Your ears are priceless. As a rough estimate, try talking at a normal volume level, as if you were talking to someone else in the room. If you can't hear your own voise very well, the music is really loud.
As for your speakers, it is very frequency dependent. The more very low frequency content, the less loud you can play them before they reach their limits. there are at least 2 ways to damage a speaker - emlting the voice coil and bottoming out the voice coil. I'm not swure there's a good way to tell your overheating the voice coil until it's too late. If you hear a buzzing sound or a loud crack on loud deep bass notes, your hitting the voice coil against the magnets, and are at risk of bending the voice coil and ruining the speaker.

When you hear distortion at a loud volume, turn it down a bit to see if you hear the same distortion. If you do, it's the recording. If you don't, you're over driving something in your system.