Sounds very much like a bad interconnect or termination. Get some contact cleaner and clean all the connections. Also, depending on what terminations you have, unscrew the outer cover on the RCA's, and check the solder joint on them. Re-solder as necessary. Mapleshade or Walker's silver paste helps make good connections stay that way, if you do that to ALL your connectors, problems of that sort will pretty much become a thing of the past. Hope that's your problem.
Good luck,
Dan
Good luck,
Dan