Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9776

HarFileSystem.listStatus() returns invalid authority if port number is empty

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.2.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If the given har URI is "har://<scheme>-localhost/usr/my.har/a", the result of HarFileSystem.listStatus() will have a ":" appended after localhost, like this: "har://<scheme>-localhost:/usr/my.har/a". it should return "har://<scheme>-localhost/usr/my.bar/a" instead.

      This creates problem when running a hive unit test TestCliDriver (archive_excludeHadoop20.q), generating the following error:

      java.io.IOException: cannot find dir = har://pfile-localhost:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har/000000_0 in pathToPartitionInfo: [pfile:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=11, har://pfile-localhost/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har]
      [junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:298)
      [junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:260)
      [junit] at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit.<init>(CombineHiveInputFormat.java:104)

        Attachments

        1. HADOOP-9776-3.patch
          2 kB
          shanyu zhao
        2. HADOOP-9776-2.patch
          2 kB
          shanyu zhao
        3. HADOOP-9776.patch
          0.8 kB
          shanyu zhao

          Issue Links

            Activity

              People

              • Assignee:
                shanyu shanyu zhao
                Reporter:
                shanyu shanyu zhao
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: