[HIVE-18319] Upgrade to Hadoop 3.0.0

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Target Version/s: 3.1.0
• Assignee: Sahil Takiar
• Reporter: Sahil Takiar
• Created: 20/Dec/17 01:24
• Updated: 17/Nov/22 08:54
• Resolved: 16/Apr/18 03:42

Description

Hadoop 3.0.0 has been released, we should upgrade to it:

http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/3.0.0/RELEASENOTES.3.0.0.html

https://hadoop.apache.org/docs/r3.0.0/index.html

Some test failures can be found when we tried to upgrade to Hadoop 3.0.0 in HIVE-17684

Issue Links

• blocks: HIVE-16295 Add support for using Hadoop's S3A OutputCommitter
• blocks: OOZIE-2973 Make sure Oozie works with Hadoop 3
• blocks: HIVE-17684 HoS memory issues with MapJoinMemoryExhaustionHandler
• is blocked by: HIVE-18858 System properties in job configuration not resolved when submitting MR job
• is blocked by: HIVE-18327 Remove the unnecessary HiveConf dependency for MiniHiveKdc
• is related to: HIVE-15016 Run tests with Hadoop 3.0.0-beta1
• is related to: HIVE-19217 Upgrade to Hadoop 3.1.0