I propose new tests that write to the same file with: less data, more data, the same amount of data -and a different set of data in each. The byte-by-byte verification will pick up any situation where the old data is somehow mixed with the new data.

git am formatted patch

the dataset() method used to create test datasets takes a broader modulus for its sets.

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12560374/HADOOP-9119.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1847//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1847//console

This message is automatically generated.

Don't think the char to String conversion method is necessary, String#format handles bytes right? Otherwise looks good.

the toChar() method takes anything >=32 and converts it to its equivalent 33 => ! ,etc, but for anything <32, maps to the hex value, so you can see the difference between expected and actual even for the low values

Comments:

1. Please add additional comments to toChar() method explaining breifly what is being done.
2. #writeAndRead() - I may not have understood the code correctly. Some comments:
   • what is the reason why you need to accumulate the error strings in result and why not just print LOG.warn() as error is encountered?
3. Variable name first_error_line - it is not related to really line right? You could just maintain the index where you encountered error right?
   • Also not sure why have len * 40 as the size of result ro why for loop is between first_error_line - 10 to first_error_line + 10. A brief description of why this is done would help understand the code better.
   • You could have array index of out of exception in the for loop. You should swap Math.min() and Math.max()
4. Some nits - Some lines have are more than 80 chars

I'll fix these. Does the 80-line limit still apply?

Does the 80-line limit still apply?

Yes.

Patch addressing Suresh's issues

+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12563740/HADOOP-9119-2.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2011//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2011//console

This message is automatically generated.

+1 for the change.

Committed the patch to branch-2 and trunk. Thank you Steve.

Integrated in Hadoop-trunk-Commit #3192 (See https://builds.apache.org/job/Hadoop-trunk-Commit/3192/)
HADOOP-9119. Add test to FileSystemContractBaseTest to verify integrity of overwritten files. Contributed by Steve Loughran. (Revision 1430387)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1430387
Files :

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java

Integrated in Hadoop-Yarn-trunk #91 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/91/)
HADOOP-9119. Add test to FileSystemContractBaseTest to verify integrity of overwritten files. Contributed by Steve Loughran. (Revision 1430387)

Result = SUCCESS
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1430387
Files :

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java

Integrated in Hadoop-Hdfs-trunk #1280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1280/)
HADOOP-9119. Add test to FileSystemContractBaseTest to verify integrity of overwritten files. Contributed by Steve Loughran. (Revision 1430387)

Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1430387
Files :

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java

Integrated in Hadoop-Mapreduce-trunk #1308 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1308/)
HADOOP-9119. Add test to FileSystemContractBaseTest to verify integrity of overwritten files. Contributed by Steve Loughran. (Revision 1430387)

Result = FAILURE
suresh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1430387
Files :

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/FileSystemContractBaseTest.java