Hive
  1. Hive
  2. HIVE-2024

In Driver.execute(), mapred.job.tracker is not restored if one of the task fails.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Query Processor
    • Labels:
      None

      Description

      If automatically one job is determined to run in local mode, and the task fails with error code not 0, mapred.job.tracker will remain to be local and might cause further problems.

      1. HIVE-2024.1.patch
        0.6 kB
        Siying Dong

        Activity

        Siying Dong created issue -
        Siying Dong made changes -
        Field Original Value New Value
        Attachment HIVE-2024.1.patch [ 12472701 ]
        Hide
        Siying Dong added a comment -

        It is a straight-forward simple fix. Maybe not a perfect one. I verified that it fixed the scenario.

        Show
        Siying Dong added a comment - It is a straight-forward simple fix. Maybe not a perfect one. I verified that it fixed the scenario.
        Siying Dong made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Siying Dong [ sdong ]
        Hide
        He Yongqiang added a comment -

        will take a look.

        Siying, can you give an example to show why we need to restore the jobtracker?

        Show
        He Yongqiang added a comment - will take a look. Siying, can you give an example to show why we need to restore the jobtracker?
        Hide
        Siying Dong added a comment -

        One case:

        set hive.exec.mode.local.auto=true;
        select transform using 'non_exist_program' from small_table;
        select count(1) from big_table;

        The second query will also be executed as local mode though it is not qualified.

        Show
        Siying Dong added a comment - One case: set hive.exec.mode.local.auto=true; select transform using 'non_exist_program' from small_table; select count(1) from big_table; The second query will also be executed as local mode though it is not qualified.
        Hide
        He Yongqiang added a comment -

        running tests.

        Show
        He Yongqiang added a comment - running tests.
        Hide
        He Yongqiang added a comment -

        Committed. Thanks Siying!

        Show
        He Yongqiang added a comment - Committed. Thanks Siying!
        He Yongqiang made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Carl Steinbach made changes -
        Fix Version/s 0.8.0 [ 12316178 ]
        Component/s Query Processor [ 12312586 ]
        Carl Steinbach made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Siying Dong
            Reporter:
            Siying Dong
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development