The answer to your original question is 'No'... or more to the point, it only makes a difference to you.
If you're trying to win an argument, no, your experience means nothing... unless you've been listening with the other person's ears.
If, OTOH, you are trying to evaluate systems & gear... it can be VERY helpful to have heard - not only the best - but a wide range of systems & components, multiple times if possible.
HOWEVER, I'd say the most important asset in evaluating audio is a lot of experience hearing live music in (many) real acoustic spaces. If you have a good sense-memory of what voices & instruments sound like, it becomes much easier to recognize how closely an audio system is reproducing that. At least it works for me.
I hope that helps.
Happy Listening!