Hive
  1. Hive
  2. HIVE-1524

parallel execution failed if mapred.job.name is set

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      The plan file name was generated based on mapred.job.name. If the user specify mapred.job.name before the query, two parallel queries will have conflict plan file name.

      1. HIVE-1524.2.patch
        39 kB
        Ning Zhang
      2. HIVE-1524.patch
        39 kB
        Ning Zhang
      3. HIVE-1524-for-Hive-0.6.patch
        37 kB
        Yuanjun Li

        Activity

        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Carl Steinbach made changes -
        Release Note Committed to branch 0.6. Thanks Yuanjun!
        Affects Version/s 0.5.0 [ 12314156 ]
        Carl Steinbach made changes -
        Fix Version/s 0.7.0 [ 12315150 ]
        Hide
        Yuanjun Li added a comment -

        I have done the load test in our company using branch 0.6 with HIVE-1524 patch, it seems ok.
        And another interesting thing is the HIVE-1019 problem will be sloved by HIVE-1524 as same time, if HIVE-1524 works well with branch 0.6

        Show
        Yuanjun Li added a comment - I have done the load test in our company using branch 0.6 with HIVE-1524 patch, it seems ok. And another interesting thing is the HIVE-1019 problem will be sloved by HIVE-1524 as same time, if HIVE-1524 works well with branch 0.6
        Ning Zhang made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Release Note Committed to branch 0.6. Thanks Yuanjun!
        Resolution Fixed [ 1 ]
        Hide
        Ning Zhang added a comment -

        Currently branch 0.6 is broken. It may be caused by HIVE-675 patch. I'll run test after that one is resolved.

        Show
        Ning Zhang added a comment - Currently branch 0.6 is broken. It may be caused by HIVE-675 patch. I'll run test after that one is resolved.
        Carl Steinbach made changes -
        Fix Version/s 0.6.0 [ 12314524 ]
        Component/s Query Processor [ 12312586 ]
        Carl Steinbach made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Carl Steinbach added a comment -

        Reopening for backport to 0.6

        Show
        Carl Steinbach added a comment - Reopening for backport to 0.6
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455826 ]
        Hide
        Yuanjun Li added a comment -

        I have re-upload this patch including parallel.q and parallel.q.out

        Show
        Yuanjun Li added a comment - I have re-upload this patch including parallel.q and parallel.q.out
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455807 ]
        Hide
        Ning Zhang added a comment -

        Thanks for back porting to 0.6 yourchanges. The code changes look good. Can you include the other 2 files (parallel.q and parallel.q.out) in the patch as well?

        Show
        Ning Zhang added a comment - Thanks for back porting to 0.6 yourchanges. The code changes look good. Can you include the other 2 files (parallel.q and parallel.q.out) in the patch as well?
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455807 ]
        Hide
        Yuanjun Li added a comment -

        HIVE-1524 backporting to HIVE 0.6 branch

        Show
        Yuanjun Li added a comment - HIVE-1524 backporting to HIVE 0.6 branch
        Joydeep Sen Sarma made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Joydeep Sen Sarma added a comment -

        committed - thanks Ning.

        Show
        Joydeep Sen Sarma added a comment - committed - thanks Ning.
        Joydeep Sen Sarma made changes -
        Affects Version/s 0.5.0 [ 12314156 ]
        Affects Version/s 0.7.0 [ 12315150 ]
        Hide
        Joydeep Sen Sarma added a comment -

        will commit once tests pass.

        Show
        Joydeep Sen Sarma added a comment - will commit once tests pass.
        Ning Zhang made changes -
        Attachment HIVE-1524.2.patch [ 12451708 ]
        Hide
        Ning Zhang added a comment -

        changed getJobID() to getHiveJobID().

        Show
        Ning Zhang added a comment - changed getJobID() to getHiveJobID().
        Hide
        Joydeep Sen Sarma added a comment -

        looks good to me.

        one comment: getJobID is a very confusing name (sounds like we are getting the hadoop jobid or something like that). it would be nice to make it more explicit (getHiveJobID perhaps?).

        Show
        Joydeep Sen Sarma added a comment - looks good to me. one comment: getJobID is a very confusing name (sounds like we are getting the hadoop jobid or something like that). it would be nice to make it more explicit (getHiveJobID perhaps?).
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Ning Zhang added a comment -

        tests are still running now.

        Show
        Ning Zhang added a comment - tests are still running now.
        Ning Zhang made changes -
        Attachment HIVE-1524.patch [ 12451640 ]
        Ning Zhang made changes -
        Attachment HIVE-1524.patch [ 12451638 ]
        Ning Zhang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ning Zhang made changes -
        Field Original Value New Value
        Attachment HIVE-1524.patch [ 12451638 ]
        Hide
        Ning Zhang added a comment -

        Using UUID to get a unique job ID for each MapRedTask and keep it in the JobConf.

        Show
        Ning Zhang added a comment - Using UUID to get a unique job ID for each MapRedTask and keep it in the JobConf.
        Ning Zhang created issue -

          People

          • Assignee:
            Ning Zhang
            Reporter:
            Ning Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development