Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9776

HarFileSystem.listStatus() returns invalid authority if port number is empty

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.1.0-beta, 3.0.0-alpha1
    • 2.2.0
    • fs
    • None
    • Reviewed

    Description

      If the given har URI is "har://<scheme>-localhost/usr/my.har/a", the result of HarFileSystem.listStatus() will have a ":" appended after localhost, like this: "har://<scheme>-localhost:/usr/my.har/a". it should return "har://<scheme>-localhost/usr/my.bar/a" instead.

      This creates problem when running a hive unit test TestCliDriver (archive_excludeHadoop20.q), generating the following error:

      java.io.IOException: cannot find dir = har://pfile-localhost:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har/000000_0 in pathToPartitionInfo: [pfile:/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=11, har://pfile-localhost/GitHub/hive-monarch/build/ql/test/data/warehouse/tstsrcpart/ds=2008-04-08/hr=12/data.har]
      [junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:298)
      [junit] at org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:260)
      [junit] at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat$CombineHiveInputSplit.<init>(CombineHiveInputFormat.java:104)

      Attachments

        1. HADOOP-9776.patch
          0.8 kB
          shanyu zhao
        2. HADOOP-9776-2.patch
          2 kB
          shanyu zhao
        3. HADOOP-9776-3.patch
          2 kB
          shanyu zhao

        Issue Links

          Activity

            People

              shanyu shanyu zhao
              shanyu shanyu zhao
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: