paranoid listener-damaging speakers?

I am one of those guys who is always wondering if he is listening too loud for his speakers capability. my system briefly consists of a prima luna prologue 2 integrated, custom eton 2 way speakers with a silk dome tweeter and 8 inch midrange, with a mhdt labs constantine dac. my room is 15x12 feet roughly. i listen about 6 feet away.
I like to listen at a level where i can feel the bass and midbass and feel that the speakers are loud enough to recreate their original acoustic on the recording. is there a rough guide to know if i am listening too loud without a meter? i will occasinally think i hear some distortion on loud passages, but it may be on the recording, i may just be paranoid? advice please? thanks.