Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3930

Decide how to integrate scheduler info into CLI and job tracker web page

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.19.0
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Changed TaskScheduler to expose API for Web UI and Command Line Tool.

      Description

      We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to provide info to display on the JobTracker web interface and in the CLI. The main things needed seem to be:

      • A way for schedulers to provide info to show in a column on the web UI and in the CLI - something as simple as a single string, or a map<string, int> for multiple parameters.
      • Some sorting order for jobs - maybe a method to sort a list of jobs.

      Let's figure out what the best way to do this is and implement it in the existing schedulers.

      My first-order proposal at an API: Augment the TaskScheduler with

      • public Map<String, String> getSchedulingInfo(JobInProgress job) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of jobs.
      • public Map<String, String> getSchedulingInfo(String queue) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of queues.
      • public Collection<JobInProgress> getJobs(String queueName) – returns the list of jobs in a given queue, sorted by a scheduler-specific order (the order it wants to run them in / schedule the next task in / etc).
      • public List<String> getQueues();

        Attachments

        1. 3930-1.patch
          12 kB
          Sreekanth Ramakrishnan
        2. HADOOP-3930-10.patch
          60 kB
          Sreekanth Ramakrishnan
        3. HADOOP-3930-11.patch
          61 kB
          Sreekanth Ramakrishnan
        4. HADOOP-3930-2.patch
          38 kB
          Sreekanth Ramakrishnan
        5. HADOOP-3930-3.patch
          46 kB
          Sreekanth Ramakrishnan
        6. HADOOP-3930-4.patch
          46 kB
          Sreekanth Ramakrishnan
        7. HADOOP-3930-5.patch
          48 kB
          Sreekanth Ramakrishnan
        8. HADOOP-3930-6.patch
          48 kB
          Sreekanth Ramakrishnan
        9. HADOOP-3930-7.patch
          57 kB
          Sreekanth Ramakrishnan
        10. HADOOP-3930-8.patch
          63 kB
          Sreekanth Ramakrishnan
        11. HADOOP-3930-9.patch
          63 kB
          Sreekanth Ramakrishnan
        12. mockup.JPG
          175 kB
          Sreekanth Ramakrishnan

          Issue Links

            Activity

              People

              • Assignee:
                sreekanth Sreekanth Ramakrishnan
                Reporter:
                matei@eecs.berkeley.edu Matei Zaharia
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: