Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4289

JobStatus.getReduceProgress() and JobStatus.getMapProgress() API's not giving any values

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      1.Run a simple job
      2.Invoke JobStatus.getReduceProgress() and JobStatus.getMapProgress() API's

      Observe that these API's are giving zeros instead of showing map/reduce progress

        Activity

        Hide
        Devaraj K added a comment -

        It is coming due to the hard-coded values in the below code,

        TypeConverter.java
        public static JobStatus fromYarn(ApplicationReport application,
              String jobFile) {
            String trackingUrl = application.getTrackingUrl();
            trackingUrl = trackingUrl == null ? "" : trackingUrl;
            JobStatus jobStatus =
              new JobStatus(
                  TypeConverter.fromYarn(application.getApplicationId()),
                  0.0f, 0.0f, 0.0f, 0.0f,
                  TypeConverter.fromYarn(application.getYarnApplicationState(), 
        	  application.getFinalApplicationStatus()),
                  org.apache.hadoop.mapreduce.JobPriority.NORMAL,
                  application.getUser(), application.getName(),
                  application.getQueue(), jobFile, trackingUrl, false
              );
            jobStatus.setSchedulingInfo(trackingUrl); // Set AM tracking url
        
        

        Here we don't have any provision to get the map and reduce progresses from RM.

        It doesn't look good to show these hard-coded values always to the user when they use JobClient.getAllJobs() API.

        Any thoughts?

        Show
        Devaraj K added a comment - It is coming due to the hard-coded values in the below code, TypeConverter.java public static JobStatus fromYarn(ApplicationReport application, String jobFile) { String trackingUrl = application.getTrackingUrl(); trackingUrl = trackingUrl == null ? "" : trackingUrl; JobStatus jobStatus = new JobStatus( TypeConverter.fromYarn(application.getApplicationId()), 0.0f, 0.0f, 0.0f, 0.0f, TypeConverter.fromYarn(application.getYarnApplicationState(), application.getFinalApplicationStatus()), org.apache.hadoop.mapreduce.JobPriority.NORMAL, application.getUser(), application.getName(), application.getQueue(), jobFile, trackingUrl, false ); jobStatus.setSchedulingInfo(trackingUrl); // Set AM tracking url Here we don't have any provision to get the map and reduce progresses from RM. It doesn't look good to show these hard-coded values always to the user when they use JobClient.getAllJobs() API. Any thoughts?
        Hide
        Karthik Kambatla (Inactive) added a comment -

        Marking as unassigned, should anyone be interested in working on it.

        Will come back to this if still available when time permits.

        Show
        Karthik Kambatla (Inactive) added a comment - Marking as unassigned, should anyone be interested in working on it. Will come back to this if still available when time permits.

          People

          • Assignee:
            Unassigned
            Reporter:
            Nishan Shetty
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:

              Development