I have tried it both ways- in my case a PS Audio Direct Stream DAC to an Audio Research SP 15, an Audio Research Ref 6SE and direct to amp.
My experience is this: If I don't mind having lower volume recordings preventing me from getting the DBs I want (I like 80-96DB) DAC to amp is just fine.
Alternatively my SP 15 added a certain brightness to the mid/highs which I wasn't satisfied with.
My Ref 6 SE was a big volume booster- and nearly neutral. Zero added shades. It did add discernable separation of string instruments (airness?) and the feeling I could "see" behind the cello as it played. I thought that was really cool and worth the price of admission.
I no longer use phono and CD so switching between sources was not important to me.
There is also the budget issue. DAC to amp saves you the costs of buying and maintaining a preamp. At least for me having great music and a small budget when I got back into the hobby made DAC to amp a great deal.
I hope my "eye witness" experience helps you with your decision.