[BIGTOP-3708] Fix build failure of Hive against Hadoop 3.3.3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0
• Assignee: Masatake Iwasaki
• Reporter: Leona Yoda
• Created: 23/Jun/22 07:35
• Updated: 21/Jul/22 09:10
• Resolved: 21/Jul/22 09:10

Description

Building Hive with Hadoop 3.3.3 failed

[ERROR] /home/ubuntu/bigtop-dev-2/bigtop/output/hive/hive-3.1.3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[31,68] org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueuePlacementPolicy is not public in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair; cannot be accessed from outside package
[ERROR] /home/ubuntu/bigtop-dev-2/bigtop/output/hive/hive-3.1.3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[43,48] no suitable constructor found for AllocationFileLoaderService(no arguments)
    constructor org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.AllocationFileLoaderService(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler) is not applicable
      (actual and formal argument lists differ in length)
    constructor org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.AllocationFileLoaderService(org.apache.hadoop.yarn.util.Clock,org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler) is not applicable
      (actual and formal argument lists differ in length)
[ERROR] /home/ubuntu/bigtop-dev-2/bigtop/output/hive/hive-3.1.3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[57,49] incompatible types: org.apache.hadoop.conf.Configuration cannot be converted to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler
[ERROR] /home/ubuntu/bigtop-dev-2/bigtop/output/hive/hive-3.1.3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[59,5] cannot find symbol
  symbol:   class QueuePlacementPolicy
  location: class org.apache.hadoop.hive.schshim.FairSchedulerShim
[ERROR] /home/ubuntu/bigtop-dev-2/bigtop/output/hive/hive-3.1.3/shims/scheduler/src/main/java/org/apache/hadoop/hive/schshim/FairSchedulerShim.java:[59,55] cannot find symbol
  symbol:   method getPlacementPolicy()
  location: class org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationConfiguration

 

 

 

Issue Links

• is related to: HIVE-22398 Remove Yarn queue management via ShimLoader.
• links to: GitHub pull request #958