SB Touch to Metrum Octave: toslink, coax RCA, BNC?

There are 75ohm RCA plugs. Good coax should be better than glass Toslink but I'm sure you'll find people with exactly opposite experience. It is system dependent.

Coax is sensitive to impedance mismatch and ambient electrical noise.

Toslink does not create ground loops but has slower transitions making it more sensitive to system noise.

Other people experience doesn't count, IMHO, since nobody has exactly the same system with the same noise, house wiring, area electrical noise etc.

I used Canare 75ohm RCA plugs.

It wasn't Octave, just short coax between CDP and DAC without changing female socket. I used Canare coax and connectors but there is a lot of different choices. Quality of shielding is extremely important. Buy or build either very short <12" or >1.5m because of transmission line effects (reflections in the cable affecting edges).

Connection becomes transmission line (relections on impedance boundaries) when propagation time is less than 1/8 of transition time. Signal travels about 1.5ns/ft (5ns/m) allowing for 8x1.5=12ns transitions. Typical transport has 25ns transitions - we're safe.

For 1.5m cable reflections might happen, but they will just miss original edge. Reflection starts from the beginning of transition and takes with 1.5m cable (both ways) 15ns to come back (3mx5ns/m) and add to original edge and it's just missed threshold point that is roughly in the middle of transition (12.5ns).