The attached patches fix the issue in 3.3/3.4/trunk. Notice:
1) the core fix is in FileTxnSnapLog - specifically reopening the snap/txnlog files after doing the truncation. I would have liked to have just closed and created the FTSL, however there were multiple objects holding references to this object, not just ZKDB. I looked at changing this however the impact was significant - it could also have repercussions on third parties embedding the zkserver. In the end I went for the reset approach.
2) notice that the code fix is relatively small. The tests make up the bulk of the changes.
3) I've tested this for all three branches successfully.
4) I added a truncate test that verified that the append after truncate worked properly. This test failed prior to applying the fix.
5) I also harvested the core test from Christian's original attachment (awesome!). This allowed me to verify and include tests that will verify this doesn't get broken again. I also verified that without applying the fix this test will fail.
6) bonus is the addition of PortForwarder - this should allow us to easily expand testing which introduces network failures of various types.