Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-1969

TestMinimrCliDriver merge_dynamic_partition2 and 3 are failing on trunk

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.7.0
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I haven't looked into it yet but saw this at the end of the .q.out:

      +Ended Job = job_201102071402_0020 with errors
      +FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask

        Activity

        Hide
        jvs John Sichi added a comment -

        Committed. Thanks Ning!

        Show
        jvs John Sichi added a comment - Committed. Thanks Ning!
        Hide
        jvs John Sichi added a comment -

        +1. Will commit.

        Show
        jvs John Sichi added a comment - +1. Will commit.
        Hide
        nzhang Ning Zhang added a comment -

        There are 2 things that make these tests failed:
        1) the recent change of default file format from HiveInputFormat to CombineHiveInputFormat makes the merge job submission fail. The current mini HDFS implementation used in minimr doesn't support CombineFileInputFormat.

        2) even though I changed to input format to HiveInputFormat and make the merge using a mapreduce job rather than map-only job, it still failed because of a bug that exist in that code path. HIVE-1980 was filed for that.

        For now we should remove merge_dynamic_partition[23].q from minimr.query.files since otherwise they won't be tested in regular TestCliDriver. I will add the equivalent test cases to minimr test in HIVE-1980.

        Show
        nzhang Ning Zhang added a comment - There are 2 things that make these tests failed: 1) the recent change of default file format from HiveInputFormat to CombineHiveInputFormat makes the merge job submission fail. The current mini HDFS implementation used in minimr doesn't support CombineFileInputFormat. 2) even though I changed to input format to HiveInputFormat and make the merge using a mapreduce job rather than map-only job, it still failed because of a bug that exist in that code path. HIVE-1980 was filed for that. For now we should remove merge_dynamic_partition [23] .q from minimr.query.files since otherwise they won't be tested in regular TestCliDriver. I will add the equivalent test cases to minimr test in HIVE-1980 .
        Hide
        nzhang Ning Zhang added a comment -

        I will take a look.

        Show
        nzhang Ning Zhang added a comment - I will take a look.

          People

          • Assignee:
            nzhang Ning Zhang
            Reporter:
            jvs John Sichi
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development