Hive
  1. Hive
  2. HIVE-254

Tracker page should provide a link to the "next step"

    Details

    • Type: Wish Wish
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: Web UI
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This may belong in Hadoop Core, or maybe more about Hive integration with Hadoop, but many queries these days take multiple steps or mapreduce phases. It would be lovely if a "finished" mapreduce from a Hive query that has several steps could provide a link to the tracker page for the next mapreduce step.

      1. click_through.png
        92 kB
        Edward Capriolo
      2. hive-254.diff
        3 kB
        Edward Capriolo

        Activity

        Hide
        Edward Capriolo added a comment -

        The hive history page has been added. I can make the page hyperlink to the job tracker.

        Show
        Edward Capriolo added a comment - The hive history page has been added. I can make the page hyperlink to the job tracker.
        Hide
        Edward Capriolo added a comment -

        This patch allows hot links from the history page to the hadoop job tracker. I also corrected the hive-default which was loaded the war from hive.war hive-hwi.war.

        Show
        Edward Capriolo added a comment - This patch allows hot links from the history page to the hadoop job tracker. I also corrected the hive-default which was loaded the war from hive.war hive-hwi.war.
        Hide
        Edward Capriolo added a comment -

        Fixed conf. Added a link from hive-history to job-tracker.

        Show
        Edward Capriolo added a comment - Fixed conf. Added a link from hive-history to job-tracker.
        Hide
        Edward Capriolo added a comment -

        screen shot of session_history. Notice how any task with the key TASK_HADOOP_ID is supplemented with a hyperlink to the Hadoop Job Tracker.

        Show
        Edward Capriolo added a comment - screen shot of session_history. Notice how any task with the key TASK_HADOOP_ID is supplemented with a hyperlink to the Hadoop Job Tracker.
        Hide
        Ashish Thusoo added a comment -

        Adam,

        I don't think that this will get into hadoop core as they probably don't want to put in hive specific code into hadoop. Having said that Edwards fix does solve this in the Hive web interface..

        Edward,

        Changes look good. Is it possible to add a test case to test the new function.

        Thanks,
        Ashish

        Show
        Ashish Thusoo added a comment - Adam, I don't think that this will get into hadoop core as they probably don't want to put in hive specific code into hadoop. Having said that Edwards fix does solve this in the Hive web interface.. Edward, Changes look good. Is it possible to add a test case to test the new function. Thanks, Ashish
        Hide
        Edward Capriolo added a comment -

        Ashish,

        The test cases are running in local mode, as a result the running job produces no TASK_HADOOP_ID and there is nothing to link to. So in local mode I do not think I can produce a test case for the function.

        09/04/22 19:56:38 INFO exec.ExecDriver: Job running in-process (local Hadoop)
        09/04/22 19:56:38 INFO mapred.FileInputFormat: Total input paths to process : 1
        
        Show
        Edward Capriolo added a comment - Ashish, The test cases are running in local mode, as a result the running job produces no TASK_HADOOP_ID and there is nothing to link to. So in local mode I do not think I can produce a test case for the function. 09/04/22 19:56:38 INFO exec.ExecDriver: Job running in-process (local Hadoop) 09/04/22 19:56:38 INFO mapred.FileInputFormat: Total input paths to process : 1
        Hide
        Ashish Thusoo added a comment -

        I suppose we can add a test case for this once the MiniMR cluster work that is being done by Namit gets done as then you would get the TASK_HADOOP_ID. Can you open a JIRA for the tests that depends on

        https://issues.apache.org/jira/browse/HIVE-204

        Otherwise,

        +1 from my side. Will run the tests and commit once the test runs are good.

        Show
        Ashish Thusoo added a comment - I suppose we can add a test case for this once the MiniMR cluster work that is being done by Namit gets done as then you would get the TASK_HADOOP_ID. Can you open a JIRA for the tests that depends on https://issues.apache.org/jira/browse/HIVE-204 Otherwise, +1 from my side. Will run the tests and commit once the test runs are good.
        Hide
        Ashish Thusoo added a comment -

        committed. Thanks Edward!!

        Show
        Ashish Thusoo added a comment - committed. Thanks Edward!!

          People

          • Assignee:
            Edward Capriolo
            Reporter:
            Adam Kramer
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development