First, decide which speaker you want because this will have the most significant affect on sound. This should be first if you want to have success. Good comment above about remembering that low efficiency speakers will require better amps and may push you out of your budget.
Second, find an integrated the synergizes well with your speaker choice.
Thirdly get your source player, the best you can afford, and then if you have some budget left over some reasonably priced power cables, interconnects, speaker cables.