Hadoop Common
  1. Hadoop Common
  2. HADOOP-3930

Decide how to integrate scheduler info into CLI and job tracker web page

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Changed TaskScheduler to expose API for Web UI and Command Line Tool.

      Description

      We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to provide info to display on the JobTracker web interface and in the CLI. The main things needed seem to be:

      • A way for schedulers to provide info to show in a column on the web UI and in the CLI - something as simple as a single string, or a map<string, int> for multiple parameters.
      • Some sorting order for jobs - maybe a method to sort a list of jobs.

      Let's figure out what the best way to do this is and implement it in the existing schedulers.

      My first-order proposal at an API: Augment the TaskScheduler with

      • public Map<String, String> getSchedulingInfo(JobInProgress job) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of jobs.
      • public Map<String, String> getSchedulingInfo(String queue) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of queues.
      • public Collection<JobInProgress> getJobs(String queueName) – returns the list of jobs in a given queue, sorted by a scheduler-specific order (the order it wants to run them in / schedule the next task in / etc).
      • public List<String> getQueues();
      1. 3930-1.patch
        12 kB
        Sreekanth Ramakrishnan
      2. mockup.JPG
        175 kB
        Sreekanth Ramakrishnan
      3. HADOOP-3930-2.patch
        38 kB
        Sreekanth Ramakrishnan
      4. HADOOP-3930-3.patch
        46 kB
        Sreekanth Ramakrishnan
      5. HADOOP-3930-4.patch
        46 kB
        Sreekanth Ramakrishnan
      6. HADOOP-3930-5.patch
        48 kB
        Sreekanth Ramakrishnan
      7. HADOOP-3930-6.patch
        48 kB
        Sreekanth Ramakrishnan
      8. HADOOP-3930-7.patch
        57 kB
        Sreekanth Ramakrishnan
      9. HADOOP-3930-8.patch
        63 kB
        Sreekanth Ramakrishnan
      10. HADOOP-3930-9.patch
        63 kB
        Sreekanth Ramakrishnan
      11. HADOOP-3930-10.patch
        60 kB
        Sreekanth Ramakrishnan
      12. HADOOP-3930-11.patch
        61 kB
        Sreekanth Ramakrishnan

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Sreekanth Ramakrishnan
              Reporter:
              Matei Zaharia
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development