[HCATALOG-626] TestReaderWriter failed with hadoop 1.1.1 due to HADOOP-7823

• Type: Test
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.4
• Component/s: test
• Labels: test

Environment

REHL

• Assignee: Unassigned
• Reporter: Bing Li
• Created: 05/Mar/13 08:43
• Updated: 27/Mar/13 00:47

Description

With hcatalog 0.4.0, TestReaderWriter failed with hadoop 1.1.1 due to HADOOP-7823 (https://issues.apache.org/jira/browse/HADOOP-7823).

It will throw NPE:
<testcase classname="org.apache.hcatalog.data.TestReaderWriter" name="test" time="6.932">
<error type="java.lang.NullPointerException">java.lang.NullPointerException
at org.apache.hadoop.fs.BufferedFSInputStream.getPos(BufferedFSInputStream.java:48)
at org.apache.hadoop.fs.FSDataInputStream.getPos(FSDataInputStream.java:41)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.readChunk(ChecksumFileSystem.java:219)
at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:237)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:189)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:158)
at java.io.DataInputStream.read(DataInputStream.java:94)
at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:176)
at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:43)
at org.apache.hcatalog.mapreduce.HCatRecordReader.nextKeyValue(HCatRecordReader.java:188)
at org.apache.hcatalog.data.transfer.impl.HCatInputFormatReader$HCatRecordItr.hasNext(HCatInputFormatReader.java:107)
at org.apache.hcatalog.data.TestReaderWriter.runsInSlave(TestReaderWriter.java:139)
at org.apache.hcatalog.data.TestReaderWriter.test(TestReaderWriter.java:104)
</error>
</testcase>

According to the change sets in HADOOP-7823, hadoop always read one extra line in next() method, which will lead this NPE.
HCatalog should handle this changes in TestReaderWriter.