[HADOOP-18724] Open file fails with NumberFormatException for S3AFileSystem

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.3.5
• Fix Version/s: 3.3.6
• Component/s: fs, fs/azure, fs/s3
• Labels: pull-request-available
• Target Version/s: 3.3.6
• Assignee: Steve Loughran
• Reporter: Ayush Saxena
• Created: 28/Apr/23 13:13
• Updated: 21/Jan/24 01:12
• Resolved: 16/May/23 12:42

Description

Saw the trace for Hive-Iceberg, was using the old client, doesn't happen once I upgraded.

Caused by: java.lang.NumberFormatException: For input string: "5783.0"
	at java.base/java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.base/java.lang.Long.parseLong(Long.java:692)
	at java.base/java.lang.Long.parseLong(Long.java:817)
	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1601)
	at org.apache.hadoop.fs.s3a.impl.OpenFileSupport.prepareToOpenFile(OpenFileSupport.java:262)
	at org.apache.hadoop.fs.s3a.S3AFileSystem.openFileWithOptions(S3AFileSystem.java:5219)
	at org.apache.hadoop.fs.FileSystem$FSDataInputStreamBuilder.build(FileSystem.java:4753)
	at org.apache.iceberg.hadoop.HadoopInputFile.newStream(HadoopInputFile.java:196)
	at org.apache.iceberg.avro.AvroIterable.newFileReader(AvroIterable.java:101)
	at org.apache.iceberg.avro.AvroIterable.getMetadata(AvroIterable.java:66)

Issue Links

• causes: HADOOP-18755 openFile builder new optLong() methods break hbase-filesystem
• is caused by: HADOOP-16202 Enhance openFile() for better read performance against object stores
• is related to: HBASE-27900 [HBOSS] Open file fails with NumberFormatException for S3AFileSystem
• relates to: HIVE-27392 Iceberg: Use String instead of Long for file length in HadoopInputFile
• links to: GitHub Pull Request #5611
• links to: GitHub Pull Request #5647
• links to: GitHub Pull Request #5650