[HBASE-5346] Fix testColumnFamilyCompression and test_TIMERANGE in TestHFileOutputFormat

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.92.0, 0.94.0
• Fix Version/s: 0.92.0, 0.94.0
• Component/s: mapreduce, test
• Hadoop Flags: Reviewed
• Assignee: Gregory Chanan
• Reporter: Gregory Chanan
• Created: 07/Feb/12 21:42
• Updated: 12/Oct/12 05:35
• Resolved: 17/Feb/12 01:07

Description

Running
mvn -Dhadoop.profile=23 test -P localTests -Dtest=org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

yields this on 0.92 (for testColumnFamilyCompression and test_TIMERANGE):
Failed tests: testColumnFamilyCompression(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): HFile for column family info-A not found

Tests in error:
test_TIMERANGE(org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat): /home/gchanan/workspace/apache92/target/test-data/276cbd0c-c771-4f81-9ba8-c464c9dd7486/test_TIMERANGE_present/_temporary/0/_temporary/_attempt_200707121733_0001_m_000000_0 (Is a directory)

The problem is that these tests make incorrect assumptions about the output of mapreduce jobs. Prior to 0.23, temporary data was in, for example:
./temporary/_attempt_0000_r_000000_0/b/1979617994050536795
Now that has changed. The correct way to get that path is based on getDefaultWorkFile.

Also, the data is not moved into the outputPath until both the Task and Job are committed.