Yes, the log code is a bit more complicated now to address a few recent issues that we found :
kafka-802: flush interval based on compressed message set
kafka-765: corrupted messages cause broker to shut down
kafka-767: message size check needs to be done after offset assignment
My main concern of not addressing this in 0.8 is that this makes it hard to do index verification in our system tests.
Indexing the first message seems to make the most intuitive sense. Yes, we do need to patch DumpLogSegments to support deep message iteration, in order to do index verification.