[HADOOP-14950] har file system throws ArrayIndexOutOfBoundsException

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Labels: newbie

Environment

CDH 5.9.2

• Assignee: Yu-Tang Lin
• Reporter: Wei-Chiu Chuang
• Created: 16/Oct/17 09:18
• Updated: 02/Oct/19 17:15

Description

When listing a har file system file, it throws an AIOOBE like the following:

$ hdfs dfs -ls har:///abc.har
-ls: Fatal internal error
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.hadoop.fs.HarFileSystem$HarStatus.<init>(HarFileSystem.java:597)
at org.apache.hadoop.fs.HarFileSystem$HarMetaData.parseMetaData(HarFileSystem.java:1201)
at org.apache.hadoop.fs.HarFileSystem$HarMetaData.access$000(HarFileSystem.java:1098)
at org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:166)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2711)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:382)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325)
at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235)
at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218)
at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:102)
at org.apache.hadoop.fs.shell.Command.run(Command.java:165)
at org.apache.hadoop.fs.FsShell.run(FsShell.java:315)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.fs.FsShell.main(FsShell.java:372)

Checking the code, it looks like the _index file in the har is mal-formed. It expects two string separately by a space in each line, and this AIOOBE is possible if the second string does not exist.

File this jira to improve the error handling of such case.