You don't have to obsess over sound quality to be an audiophile. If you care about good sound and work on a system to provide good sound, you're an audiophile, no matter what level your system is at.
You're back on the right track now. Audiophilia is about enjoying music, so keep on enjoying!
Any pursuit can become an obsession. Now that you know that watch out for it and stop yourself before a different pursuit becomes a problem. You can do a lot worse than obsessing over sound.