[FLINK-35175] HadoopDataInputStream can't compile with Hadoop 3.2.3

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.0
• Labels: pull-request-available
• Assignee: Hangxiang Yu
• Reporter: Ryan Skraba
• Created: 19/Apr/24 13:13
• Updated: 22/Apr/24 11:54
• Resolved: 22/Apr/24 01:51

Description

Unfortunately, introduced in FLINK-35045: PREADWRITEBUFFER was added in Hadoop releases 3.3.0 and 2.10.0.

It doesn't exist in flink.hadoop.version 3.2.3, which we are using in end-to-end tests.

00:23:55.093 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project flink-hadoop-fs: Compilation failure: Compilation failure: 
00:23:55.093 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[151,63] cannot find symbol
00:23:55.094 [ERROR]   symbol:   variable READBYTEBUFFER
00:23:55.094 [ERROR]   location: interface org.apache.hadoop.fs.StreamCapabilities
00:23:55.094 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[182,63] cannot find symbol
00:23:55.094 [ERROR]   symbol:   variable PREADBYTEBUFFER
00:23:55.094 [ERROR]   location: interface org.apache.hadoop.fs.StreamCapabilities
00:23:55.094 [ERROR] /home/vsts/work/1/s/flink-filesystems/flink-hadoop-fs/src/main/java/org/apache/flink/runtime/fs/hdfs/HadoopDataInputStream.java:[183,43] incompatible types: long cannot be converted to org.apache.hadoop.io.ByteBufferPool
00:23:55.094 [ERROR] -> [Help 1] 

• 1.20 compile_cron_hadoop313 https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=59012&view=logs&j=87489130-75dc-54e4-1f45-80c30aa367a3&t=73da6d75-f30d-5d5a-acbe-487a9dcff678&l=3630

Issue Links

• relates to: FLINK-35045 Introduce ForStFileSystem to support reading and writing with ByteBuffer
• links to: GitHub Pull Request #24691