v5 attached. minor changes like log lines to debug. also changed the LogRecordAdder rotate check to >=.
I think we still need to solve the segment sizing problem though. If you don't want to actually record the high water mark, how about recycling the segments we just replayed?
Basically I don't want to inflict bad performance on people who have > 500MB of unflushed data while CL "extends" itself. Once is acceptable, but every server restart is not.
I think the logic around the size cap needs some work too. We should be strict about keeping under the cap – the idea is we could be given a partition that size, so overflowing is Bad.
If we get stricter there then I think we can simplify the "high water mark" tracking: you never discard segments (since you never violate the size constraints) so the high water mark must be exactly the current number of segments.
submitCreateFreshSegment is unused. Oversight?
Why does recycle bother creating a new Segment object? (Contrawise, I think I'd prefer creating a new SequentialWriter object, to using seek here.)
Is the order in CL.shutdownBlocking correct? Seems to me it would be less dangerous to leave the allocator active until after we're done processing mutations.