Hive
  1. Hive
  2. HIVE-1969

TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I haven't looked into it yet but saw this at the end of the .q.out:

      +Ended Job = job_201102071402_0020 with errors
      +FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

        Activity

        Hide
        Ning Zhang added a comment -

        I will take a look.

        Show
        Ning Zhang added a comment - I will take a look.
        Hide
        Ning Zhang added a comment -

        There are 2 things that make these tests failed:
        1) the recent change of default file format from HiveInputFormat to CombineHiveInputFormat makes the merge job submission fail. The current mini HDFS implementation used in minimr doesn't support CombineFileInputFormat.

        2) even though I changed to input format to HiveInputFormat and make the merge using a mapreduce job rather than map-only job, it still failed because of a bug that exist in that code path. HIVE-1980 was filed for that.

        For now we should remove merge_dynamic_partition[23].q from minimr.query.files since otherwise they won't be tested in regular TestCliDriver. I will add the equivalent test cases to minimr test in HIVE-1980.

        Show
        Ning Zhang added a comment - There are 2 things that make these tests failed: 1) the recent change of default file format from HiveInputFormat to CombineHiveInputFormat makes the merge job submission fail. The current mini HDFS implementation used in minimr doesn't support CombineFileInputFormat. 2) even though I changed to input format to HiveInputFormat and make the merge using a mapreduce job rather than map-only job, it still failed because of a bug that exist in that code path. HIVE-1980 was filed for that. For now we should remove merge_dynamic_partition [23] .q from minimr.query.files since otherwise they won't be tested in regular TestCliDriver. I will add the equivalent test cases to minimr test in HIVE-1980 .
        Hide
        John Sichi added a comment -

        +1. Will commit.

        Show
        John Sichi added a comment - +1. Will commit.
        Hide
        John Sichi added a comment -

        Committed. Thanks Ning!

        Show
        John Sichi added a comment - Committed. Thanks Ning!

          People

          • Assignee:
            Ning Zhang
            Reporter:
            John Sichi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development