The starting point for profitability in retail speakers is to sell them for 10x the driver cost. Everything after this is "bonus."
Off shore manufacturing, and "in house" drivers are two ways to achieve this.
The last thing you want to do is offshore, and built in house drivers while ONLY achieving a 10:1 ratio. That way lies barely hanging on.
Custom, one person shops like Fritz are rare exceptions.
I also want to say that I think being a bespoke speaker manufacturer in this economy is one of the most challenging gigs you can have.