I second wolf_garcia’s suggestion; the Schiit Loki Mini would work really well as a "loudness" switch. Set the desired attenuation/gain for each tone control level for quiet listening and then you can use the bypass switch to go "straight wire" when you’re listening at typical volume levels. Only drawback is that with the Brio there isn’t a tape monitor feature so you’d be limited to using the Loki for only one component; or one component at a time. :-)
Regards,
Tom
PS: Open back headphones are also a good solution. It's what I used when we had a baby in the house.