[HADOOP-18887] Java 17 Runtime Support

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: build
• Flags: Important
• Assignee: Unassigned
• Reporter: Bruno Ramirez
• Created: 11/Sep/23 21:26
• Updated: 30/Oct/23 18:43

Description

This JIRA feature aims to extend the Java runtime support for Hadoop to include Java 17 in addition to Java 8 and Java 11. Currently Hadoop version 3.3.6 supports Java 8 compile and Java 8/11 runtime. The goal is to make Hadoop compatible with Java 17 runtime as well.

The plan for this release is to allow Hadoop to default to Java 11/Java 17, while still providing the flexibility for customers to configure Hadoop to use Java 8, Java 11, or Java 17 based on their specific needs. This project's objectives include:

1. Certifying that Hadoop works seamlessly on Java 8/11/17 for common use cases.
2. Ensuring that running Hadoop on Java 11/17 does not disrupt other applications and libraries such as Spark, Hive, Flink, Presto/Trino, and HBase.

The decision to support Java 17 runtime is motivated by customer requests and significant performance improvements observed in downstream applications like Apache Hive and Apache Spark. The testing process encompasses unit tests, integration tests, and performance tests, as well as verifying the proper functioning of all Hadoop daemons with Java 17.

The project will address compile time issues across various Hadoop components, ensuring that Hadoop remains compatible with Java 17 throughout the entire codebase.

This ticket serves as a vital step in enhancing Hadoop's capabilities, providing customers with more choices and improved performance for their big data processing needs.

Issue Links

• is a child of: HADOOP-17177 Java 17 support
• is a parent of: HADOOP-18898 Add argLine configuration to maven-surefire-plugin
• is a parent of: HADOOP-18899 Configure add opens in setup JVM launch command
• is a parent of: HADOOP-18900 Add JAVA 17 in hadoop-env.sh
• is a parent of: HADOOP-18901 [Java17] Create maven profile for running unit tests
• is a parent of: HADOOP-18902 KMS server not running on Java 17 by default
• is a parent of: HADOOP-18903 Added yarn.app.mapreduce.container.jvm.jre17.args as a config
• is a parent of: YARN-11602 Pass additional arguments to YARN / Child JVM containers
• requires: HADOOP-18953 Java 17 runtime library upgrade prerequisites
• links to: Oss/hadoop java17 runtime feature branch prerequisite #6020