1. I see the separation between mid-fi and high end as being more about philosophy & voicing than price. Companies like Schiit & First Watt make cosmetic & economic compromises to produce gear with more than a whiff of high end performance.
2. There is actually MORE variation in system sound in the high end. Mid fi systems tend to use the same tried-and-true cost-effective designs, whereas high end companies will go to extreme lengths to achieve their voicing standards. Horns, electrostats, planars, ribbons, field coils, tubes, GAN FETs, R2R, FGA, etc. are all capable of making significant differences in your system.
3. I also highly recommend listening to live music, for its own merits. If you love the sound of unamplified instruments, they will set the standard you seek with your sound system.
4. Many of the responses here are from those who have made a long journey. Don't be intimidated by their stories! Go listen to as many systems as you can. Buy the speakers you like best & can afford. Build your system around them and enjoy! If you like them enough, you might not need to go through another full system upgrade for decades. Of course, you probably will need to do the occasional technology upgrade.