Stick to your budget first...
Then because i am in the same situation, i suggest as i did myself to use small active speakers as another possible road... Mine are stunning but i modify them .. I paid them 100 bucks 12 years ago and i disliked them till the day i was in the obligation to use only them for music one year ago ... I created for my listening field corner an acoustic treatment and i damp this speakers heavily , i modify their rear porthole design with various straws or various size and i coupled/decoupled the box speakers from my desk to control vibrations/resonance with a sandwich of various materials : ( shungite plate-cork plate-oak plate-sorbothane plate-bamboo plate-granite plate )
The results is a complete metamorphosis with a specs going from 80 hertz to a clear not boomy 50 hertz ... Now they beat any headphone i ever own save my top one (9 of three types) not bad for a speaker i dislike for 12 years and never use for music till one year ago ...
My main headphone is driven by an independant headphone out from a big very good amplifier though and an independant battery dac (Hidizs) ...( low cost at the times all this for dac+Sansui amp 600 bucks + 100 bucks for the speakers and because i was lucky few peanuts for the french SPS french dac ).. By the way i tried few months ago to upgrade my big Sansui alpha for a modern acclaimed tube headphone amplifier and i fail and return it and i loose 300 bucks to learn that lesson ... ...All vintage amp headphone out are not all junk ... And some headphone need more than most headphone amp ... ( as the AKG K340 my main music headphone way over the others )
Then i bought a Fosi SK01 battery pre-amplifier a S.S. (65 bucks) one with precise 3 tone controls and a loudness button with two headphone entries ...And a line out for my powered speakers and a line out for ma marvellous french battery dac ...I use this for my secondary headphones (AKG k701 or Fostex th 7 )
To beat my two systems we must invest ten times the amount at least ...Dont think ever that my low cost system dont sound more than just good ...( i modified my headphone too for sure)
The difference between a good or a bad system if the gear pieces are compatible and relatively good is the three working dimensions controls well done or not : electrical noise floor, mechanical vibration control of the speakers and gear and especially acoustic mechanical control of the room even in near listening ...
Stick to your budget and think about the way to embed rightfully your system ...There is no comparison between my system right out of the box and after the three embeddings controls installed.. NONE ...
For sure you must pick the right components to start with ...
The rest is not a question of money but of embeddings controls ..
Stick to your budget, anyway there is always costlier better gear which will be proposed here without ceiling and without end but there is ONLY ONE WAY TO EMBED RIGHTFULLY ANY SYSTEM... learn it before buying anything very costly ......
It is my double advices : stick to your budget and learn how to install it in your room ... Anybody can suggest a costlier piece of gear, nothing is easier than suggesting always costlier gear upgrade but what matter is learning how to embed what we have already electrically, mechanically and acoustically ...