It's two-fold.
When the music sounds spectacular, I am done. I just want to listen.
When I see a gear and start thinking about what it might do for me, I am back to being not done.
How good is good enough?
Most of us here cannot afford six figure prices for each component (assuming that will bring the best sound.) So how far do we want to go to improve our systems? There are always bigger fish. When does it stop? It stops when we say it stops, when our gear brings us satisfaction. To constantly strive for better sound is an endless quest, not necessarily based on the quality of our set but on our personality.