[HADOOP-9776] HarFileSystem.listStatus() returns invalid authority if port number is empty

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
• Fix Version/s: 2.2.0
• Component/s: fs
• Hadoop Flags: Reviewed
• Assignee: shanyu zhao
• Reporter: shanyu zhao
• Created: 26/Jul/13 01:22
• Updated: 12/May/16 18:24
• Resolved: 25/Sep/13 05:32

Description

If the given har URI is "har://<scheme>-localhost/usr/my.har/a", the result of HarFileSystem.listStatus() will have a ":" appended after localhost, like this: "har://<scheme>-localhost:/usr/my.har/a". it should return "har://<scheme>-localhost/usr/my.bar/a" instead.

This creates problem when running a hive unit test TestCliDriver (archive_excludeHadoop20.q), generating the following error:

java.io.IOException: cannot find dir = har://pfile-localhost:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har/000000_0 in pathToPartitionInfo: [pfile:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=11, har://pfile-localhost/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har]
[junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:298)
[junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:260)
[junit] at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit.<init>(CombineHiveInputFormat.java:104)

Issue Links

• is related to: HIVE-8034 Don't add colon when no port is specified