[HIVE-16860] HostUtil.getTaskLogUrl change between hadoop 2.3 and 2.4 breaks at runtime.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 0.14.0
• Component/s: Shims
• Assignee: Jason Dere
• Reporter: Chris Drome
• Created: 08/Jun/17 18:29
• Updated: 08/Jun/17 18:43
• Resolved: 08/Jun/17 18:43

Description

Original issue in HIVE-6900 :

The signature for HostUtil.getTaskLogUrl has changed between Hadoop-2.3 and Hadoop-2.4.

Code in shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java works with Hadoop-2.3 method and causes compilation failure with Hadoop-2.4.

The issue with HIVE-6900 is still not completely fixed, in that compilation succeeds, but this fails now at runtime because https://issues.apache.org/jira/secure/attachment/12651497/MAPREDUCE-5830.patch was committed in hadoop - The result is that our reflection check to see if the 23_METHOD exists succeeds, which means 23_METHOD will not be null, and 24_METHOD will be null, and thus, at runtime, we will wind up calling 23_METHOD. If hadoop had not "fixed" this bug, with this fix, our shim works, but with their fix, it looks like we still have a runtime problem.

Issue Links

• blocks: HIVE-6960 Set Hive pom to use Hadoop-2.4
• is a clone of: HIVE-6900 HostUtil.getTaskLogUrl signature change causes compilation to fail
• is related to: MAPREDUCE-5857 Need task log URL API for external users