Hi Greh, I had the same problem as you with an old Audiolab 8000s amplifier. In my oppinion (and I would say I am pretty sure) it is a potentiometer problem. I sent the unit for repair, and they found a small imbalance between left and right channel that came from the potentiometer (ALPS blue pot). They changed it but the imbalance is still obvious and not only in the lowest listening level but at almost all volume settings (and yes, I paid the ridiculously high amount of $300, 150 of which were of course refunded). I only use one source and so I built a small passive preamp that I now plug directly to the power amp section and the problem is solved completely. Of course there are many other drawbacks with the passive (but also some advantages...).
If you do not want to go this way, what you could do is to change the potentiometer inside your amp with a better one, or an attenuator (i.e.DUCT) at best. I would be pretty sure that your problem (both imbalance and cracking noise) will be eliminated. Let me know if you need more info and good luck...