Hive
  1. Hive
  2. HIVE-1524

parallel execution failed if mapred.job.name is set

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      The plan file name was generated based on mapred.job.name. If the user specify mapred.job.name before the query, two parallel queries will have conflict plan file name.

      1. HIVE-1524-for-Hive-0.6.patch
        37 kB
        Yuanjun Li
      2. HIVE-1524.2.patch
        39 kB
        Ning Zhang
      3. HIVE-1524.patch
        39 kB
        Ning Zhang

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        2m 54s 1 Ning Zhang 10/Aug/10 07:47
        Open Open Patch Available Patch Available
        28m 32s 2 Ning Zhang 10/Aug/10 08:03
        Patch Available Patch Available Resolved Resolved
        16h 10m 1 Joydeep Sen Sarma 11/Aug/10 00:13
        Resolved Resolved Reopened Reopened
        48d 21h 31m 1 Carl Steinbach 28/Sep/10 21:44
        Reopened Reopened Resolved Resolved
        8h 38m 1 Ning Zhang 29/Sep/10 06:23
        Resolved Resolved Closed Closed
        443d 18h 40m 1 Carl Steinbach 17/Dec/11 00:03
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Carl Steinbach made changes -
        Release Note Committed to branch 0.6. Thanks Yuanjun!
        Affects Version/s 0.5.0 [ 12314156 ]
        Carl Steinbach made changes -
        Fix Version/s 0.7.0 [ 12315150 ]
        Hide
        Yuanjun Li added a comment -

        I have done the load test in our company using branch 0.6 with HIVE-1524 patch, it seems ok.
        And another interesting thing is the HIVE-1019 problem will be sloved by HIVE-1524 as same time, if HIVE-1524 works well with branch 0.6

        Show
        Yuanjun Li added a comment - I have done the load test in our company using branch 0.6 with HIVE-1524 patch, it seems ok. And another interesting thing is the HIVE-1019 problem will be sloved by HIVE-1524 as same time, if HIVE-1524 works well with branch 0.6
        Ning Zhang made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Release Note Committed to branch 0.6. Thanks Yuanjun!
        Resolution Fixed [ 1 ]
        Hide
        Ning Zhang added a comment -

        Currently branch 0.6 is broken. It may be caused by HIVE-675 patch. I'll run test after that one is resolved.

        Show
        Ning Zhang added a comment - Currently branch 0.6 is broken. It may be caused by HIVE-675 patch. I'll run test after that one is resolved.
        Carl Steinbach made changes -
        Fix Version/s 0.6.0 [ 12314524 ]
        Component/s Query Processor [ 12312586 ]
        Carl Steinbach made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Carl Steinbach added a comment -

        Reopening for backport to 0.6

        Show
        Carl Steinbach added a comment - Reopening for backport to 0.6
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455826 ]
        Hide
        Yuanjun Li added a comment -

        I have re-upload this patch including parallel.q and parallel.q.out

        Show
        Yuanjun Li added a comment - I have re-upload this patch including parallel.q and parallel.q.out
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455807 ]
        Hide
        Ning Zhang added a comment -

        Thanks for back porting to 0.6 yourchanges. The code changes look good. Can you include the other 2 files (parallel.q and parallel.q.out) in the patch as well?

        Show
        Ning Zhang added a comment - Thanks for back porting to 0.6 yourchanges. The code changes look good. Can you include the other 2 files (parallel.q and parallel.q.out) in the patch as well?
        Yuanjun Li made changes -
        Attachment HIVE-1524-for-Hive-0.6.patch [ 12455807 ]
        Hide
        Yuanjun Li added a comment -

        HIVE-1524 backporting to HIVE 0.6 branch

        Show
        Yuanjun Li added a comment - HIVE-1524 backporting to HIVE 0.6 branch
        Joydeep Sen Sarma made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Joydeep Sen Sarma added a comment -

        committed - thanks Ning.

        Show
        Joydeep Sen Sarma added a comment - committed - thanks Ning.
        Joydeep Sen Sarma made changes -
        Affects Version/s 0.5.0 [ 12314156 ]
        Affects Version/s 0.7.0 [ 12315150 ]
        Hide
        Joydeep Sen Sarma added a comment -

        will commit once tests pass.

        Show
        Joydeep Sen Sarma added a comment - will commit once tests pass.
        Ning Zhang made changes -
        Attachment HIVE-1524.2.patch [ 12451708 ]
        Hide
        Ning Zhang added a comment -

        changed getJobID() to getHiveJobID().

        Show
        Ning Zhang added a comment - changed getJobID() to getHiveJobID().
        Hide
        Joydeep Sen Sarma added a comment -

        looks good to me.

        one comment: getJobID is a very confusing name (sounds like we are getting the hadoop jobid or something like that). it would be nice to make it more explicit (getHiveJobID perhaps?).

        Show
        Joydeep Sen Sarma added a comment - looks good to me. one comment: getJobID is a very confusing name (sounds like we are getting the hadoop jobid or something like that). it would be nice to make it more explicit (getHiveJobID perhaps?).
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Ning Zhang added a comment -

        tests are still running now.

        Show
        Ning Zhang added a comment - tests are still running now.
        Ning Zhang made changes -
        Attachment HIVE-1524.patch [ 12451640 ]
        Ning Zhang made changes -
        Attachment HIVE-1524.patch [ 12451638 ]
        Ning Zhang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Ning Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Ning Zhang made changes -
        Field Original Value New Value
        Attachment HIVE-1524.patch [ 12451638 ]
        Hide
        Ning Zhang added a comment -

        Using UUID to get a unique job ID for each MapRedTask and keep it in the JobConf.

        Show
        Ning Zhang added a comment - Using UUID to get a unique job ID for each MapRedTask and keep it in the JobConf.
        Ning Zhang created issue -

          People

          • Assignee:
            Ning Zhang
            Reporter:
            Ning Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development