According to Uptone the ground matters if you have several connections on the A side. Do you?
Tweaking Etherregen
Two tips for Etherregen users: other than using good power and ethernet cables, ideally with a 10m clock:
Always run a separate ground wire, it really matters
Stick an Akiko Audio 3D tuning chip on top of the B side low noise section (0n the casing, not the board). It is located between the ‘clean’ RJ45 socket and the ground lug. (akikoaudio.com, €59).
Let me know how you get on.