Thanks Stack and J-D. I started on having ReplicationSource manage their own logs. So far it does not actually make the code nicer and easier to read, the version I have so far also fails TestReplication. So that's for another jira (as you say).
One thing I did was to remove HServerAddress from ReplicationSource in favor of using ServerName.
HServerAddress resolves hostnames right away, which is good in this case, but as HConnectionManager caches the connection anyway, that should not be a problem.
I'll add more tests and also do real world testing, and then send an update.