Details
-
Bug
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
3.0.0
-
None
-
None
-
CDH 5.9.2
Description
When listing a har file system file, it throws an AIOOBE like the following:
$ hdfs dfs -ls har:///abc.har -ls: Fatal internal error java.lang.ArrayIndexOutOfBoundsException: 1 at org.apache.hadoop.fs.HarFileSystem$HarStatus.<init>(HarFileSystem.java:597) at org.apache.hadoop.fs.HarFileSystem$HarMetaData.parseMetaData(HarFileSystem.java:1201) at org.apache.hadoop.fs.HarFileSystem$HarMetaData.access$000(HarFileSystem.java:1098) at org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:166) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2711) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:382) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at org.apache.hadoop.fs.shell.PathData.expandAsGlob(PathData.java:325) at org.apache.hadoop.fs.shell.Command.expandArgument(Command.java:235) at org.apache.hadoop.fs.shell.Command.expandArguments(Command.java:218) at org.apache.hadoop.fs.shell.FsCommand.processRawArguments(FsCommand.java:102) at org.apache.hadoop.fs.shell.Command.run(Command.java:165) at org.apache.hadoop.fs.FsShell.run(FsShell.java:315) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) at org.apache.hadoop.fs.FsShell.main(FsShell.java:372)
Checking the code, it looks like the _index file in the har is mal-formed. It expects two string separately by a space in each line, and this AIOOBE is possible if the second string does not exist.
File this jira to improve the error handling of such case.