You should probably post this question in the digital or cable forum as well.
It has been my experience that if you are looking to take advantage of a surround processor then optical is best. If you are looking for the best 2 channel music, then I say go with the RCA if that is what your player has. That said, I have found the best digital playback is when the designer keeps the entire circuit balanced from the dacs to the preamp. IMHO, YMMV