Why have capacitors improved so much over the years?

For the interested:
Picking Capacitors - Walter G. Jung and Richard Marsh (reliablecapacitors.com)
No affiliation.

Everything has improved over the years.
Defects introduce colorations.
Some prefer the colorations.

Engineers who designed professional gear 40 and 50 years ago cared. Back then I knew some then 'old timers' who were aware. Can't comment on earlier, but no doubt some were aware.

I had a tube 'guru' tweak my Citation II with better resistors, caps & wire. Same cct & old tubes, but OMG.

While the referenced page is extensive, it's also a lot of hooey as components perform in concert with their containing circuit. There is no absolute and cost is seldom an arbiter of quality.

More than once, "Wow!" devolved

If there were a way to eliminate capacitors, that would be miraculous.

There is. They are called oscillators.

So much blather. So little knowledge.

I didn’t realize that any crossover destroys the signal.

Not all. Time correction can be done, but it requires thinking "outside the box"

Nearly 50 years ago Dahlquist did the DQ-10. Nearly 40, John Bau did the Spica TC-50 and later the Angelus. Also Wilson, Van der Steen, ... All the crossovers were complex and none of them had drivers mounted on a flat panel.

No passive crossover in a flat panel box will ever equal equal phase aligned drivers each driven by a suitable amplifier in turn driven from a bespoke active crossover. To think otherwise is kidding yourself.

Where are they used?

Did you verify values?

Please detail evaluation.

Please note that you appear to be a dealer of some sort:
"As a dealer I do realize the challenge it presents when you attempt to set up a high end system in a store environment. That is why our high end reference system is actually set up in a home environment." in  Audiogon Discussion Forum

I can't understand either why manufacturers go through all the trouble of designing a piece of gear & then at the most critical point insert cheap caps.

For one thing, a large manufacturer might pay a dollar for a cap which you might pay ten.

Adding $100 in parts cost is $500 out the door and $1000 in the store.

Many of the improvements effected by the great unwashed may be seen as detrimental by the designer in terms of sonics, reliability, etc.

An engineer, be it an all out, no holds barred or LoFi PoS has a specification of what it to be achieved and a budget to hold. Better caps could result in a flimsy chassis, under-sized heat sinks, lower quality PCB, cheaper switches, etc. etc.

There are plenty of Best of Everything manufacturers and their prices reflect it.

How many Joe HiFi tweaks are verified in engineering terms? Joe may think it better, but most may not.

Roger designed circuits requiring perfectly matched output tubes

This may be more marketing than fact. Matching to 5% is really hard and does not last much after the tube is put in service. 10% is considered pretty darned good. Industry standard used to be 20-25%.

I'm well familiar with Modjeski's brilliance. I'm pretty sure he never claimed perfect matching.

I'm not debating matching tubes. Been doing it for decades. However, if you stick a matched quad in something like a CII or ST-70, bias it up, play it a while and then re-measure the tubes, they are no longer what they were and the ratios of the various parameters have all drifted slightly.

Quality tubes will likely still qualify as a matched quad, but 2x matched pairs are within the probability realm. As is a matched trio... and an orphan.