[HIVE-17133] NoSuchMethodError in Hadoop FileStatus.compareTo

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Assignee: Rui Li
• Reporter: Rui Li
• Created: 20/Jul/17 10:14
• Updated: 22/Jun/21 12:53

Description

The stack trace is:

Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FileStatus.compareTo(Lorg/apache/hadoop/fs/FileStatus;)I
	at org.apache.hadoop.hive.ql.io.AcidUtils.lambda$getAcidState$0(AcidUtils.java:931)
	at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
	at java.util.TimSort.sort(TimSort.java:234)
	at java.util.Arrays.sort(Arrays.java:1512)
	at java.util.ArrayList.sort(ArrayList.java:1454)
	at java.util.Collections.sort(Collections.java:175)
	at org.apache.hadoop.hive.ql.io.AcidUtils.getAcidState(AcidUtils.java:929)

I'm on Hive master and using Hadoop 2.7.2. The method signature in Hadoop 2.7.2 is:
https://github.com/apache/hadoop/blob/release-2.7.2-RC2/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L336
In Hadoop 2.8.0 it becomes:
https://github.com/apache/hadoop/blob/release-2.8.0-RC3/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileStatus.java#L332
I think that breaks binary compatibility.

Issue Links

• is related to: HADOOP-12209 Comparable type should be in FileStatus
• relates to: HADOOP-14683 FileStatus.compareTo binary compatible issue