Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3930

Decide how to integrate scheduler info into CLI and job tracker web page

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.19.0
    • 0.19.0
    • None
    • None
    • Reviewed
    • Changed TaskScheduler to expose API for Web UI and Command Line Tool.

    Description

      We need a way for job schedulers such as HADOOP-3445 and HADOOP-3476 to provide info to display on the JobTracker web interface and in the CLI. The main things needed seem to be:

      • A way for schedulers to provide info to show in a column on the web UI and in the CLI - something as simple as a single string, or a map<string, int> for multiple parameters.
      • Some sorting order for jobs - maybe a method to sort a list of jobs.

      Let's figure out what the best way to do this is and implement it in the existing schedulers.

      My first-order proposal at an API: Augment the TaskScheduler with

      • public Map<String, String> getSchedulingInfo(JobInProgress job) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of jobs.
      • public Map<String, String> getSchedulingInfo(String queue) – returns key-value pairs which are displayed in columns on the web UI or the CLI for the list of queues.
      • public Collection<JobInProgress> getJobs(String queueName) – returns the list of jobs in a given queue, sorted by a scheduler-specific order (the order it wants to run them in / schedule the next task in / etc).
      • public List<String> getQueues();

      Attachments

        1. mockup.JPG
          175 kB
          Sreekanth Ramakrishnan
        2. HADOOP-3930-9.patch
          63 kB
          Sreekanth Ramakrishnan
        3. HADOOP-3930-8.patch
          63 kB
          Sreekanth Ramakrishnan
        4. HADOOP-3930-7.patch
          57 kB
          Sreekanth Ramakrishnan
        5. HADOOP-3930-6.patch
          48 kB
          Sreekanth Ramakrishnan
        6. HADOOP-3930-5.patch
          48 kB
          Sreekanth Ramakrishnan
        7. HADOOP-3930-4.patch
          46 kB
          Sreekanth Ramakrishnan
        8. HADOOP-3930-3.patch
          46 kB
          Sreekanth Ramakrishnan
        9. HADOOP-3930-2.patch
          38 kB
          Sreekanth Ramakrishnan
        10. HADOOP-3930-11.patch
          61 kB
          Sreekanth Ramakrishnan
        11. HADOOP-3930-10.patch
          60 kB
          Sreekanth Ramakrishnan
        12. 3930-1.patch
          12 kB
          Sreekanth Ramakrishnan

        Issue Links

          Activity

            People

              sreekanth Sreekanth Ramakrishnan
              matei@eecs.berkeley.edu Matei Zaharia
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: