Looks good. 4 things: +1 modulo the following
1) Change the comment
// Delete new replica.
// mark new replica as corrupt
2) for each of the cases check to see if the lease is open. If lease is not open, log an error that we got a length mismatch even when the file was not open.
Also file a jira for the case when the lease is not open to perhaps write to edits log to record the new length (I am not sure if writing the new length
is right or wrong but we can debate this on that jira).
3) Your fix will not let us distinguish between true corruption caused by some bug in HDFS, and the normal mismatch that can occur during appends when
a client dies (I am not sure of this but that is my recollection from the append discussions with Dhruba last year at yahoo).
This is okay for now. But let us file a jira to fix this so that we can distinguish.
The easy code fix for this is to add a field to internal data structure to record the original length in fsimage - but this will increase the usage of memory in the system
since the 4 bytes will be multiplied by the number of logical blocks in the system.
4) In my opinion the correct behavior for shorter blocks (but longer than the fsimage recorded length) is to invalidate as in the original code - however our invalidation code does not handle the case because if the "corrupt" block is the last one it keeps it as valid. Thus your patch is a good emergency fix to this very critical problem.
I suggest that we file a jira to handle invaliding such invalid blocks correctly.
Note here I am distinguishing between corrupt blocks (caused by hardware errors or by bugs in our software) and invalid blocks (those lenght mismatches that
can occur due to client or other failures). Others may not share the distinction I make - lets debate that in the jira; we need to get this patch out ASAP.