Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4289

JobStatus.getReduceProgress() and JobStatus.getMapProgress() API's not giving any values

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: mrv2
    • Labels:
      None

      Description

      1.Run a simple job
      2.Invoke JobStatus.getReduceProgress() and JobStatus.getMapProgress() API's

      Observe that these API's are giving zeros instead of showing map/reduce progress

        Issue Links

          Activity

          Hide
          Devaraj K added a comment -

          It is coming due to the hard-coded values in the below code,

          TypeConverter.java
          public static JobStatus fromYarn(ApplicationReport application,
                String jobFile) {
              String trackingUrl = application.getTrackingUrl();
              trackingUrl = trackingUrl == null ? "" : trackingUrl;
              JobStatus jobStatus =
                new JobStatus(
                    TypeConverter.fromYarn(application.getApplicationId()),
                    0.0f, 0.0f, 0.0f, 0.0f,
                    TypeConverter.fromYarn(application.getYarnApplicationState(), 
          	  application.getFinalApplicationStatus()),
                    org.apache.hadoop.mapreduce.JobPriority.NORMAL,
                    application.getUser(), application.getName(),
                    application.getQueue(), jobFile, trackingUrl, false
                );
              jobStatus.setSchedulingInfo(trackingUrl); // Set AM tracking url
          
          

          Here we don't have any provision to get the map and reduce progresses from RM.

          It doesn't look good to show these hard-coded values always to the user when they use JobClient.getAllJobs() API.

          Any thoughts?

          Show
          Devaraj K added a comment - It is coming due to the hard-coded values in the below code, TypeConverter.java public static JobStatus fromYarn(ApplicationReport application, String jobFile) { String trackingUrl = application.getTrackingUrl(); trackingUrl = trackingUrl == null ? "" : trackingUrl; JobStatus jobStatus = new JobStatus( TypeConverter.fromYarn(application.getApplicationId()), 0.0f, 0.0f, 0.0f, 0.0f, TypeConverter.fromYarn(application.getYarnApplicationState(), application.getFinalApplicationStatus()), org.apache.hadoop.mapreduce.JobPriority.NORMAL, application.getUser(), application.getName(), application.getQueue(), jobFile, trackingUrl, false ); jobStatus.setSchedulingInfo(trackingUrl); // Set AM tracking url Here we don't have any provision to get the map and reduce progresses from RM. It doesn't look good to show these hard-coded values always to the user when they use JobClient.getAllJobs() API. Any thoughts?
          Hide
          Karthik Kambatla (Inactive) added a comment -

          Marking as unassigned, should anyone be interested in working on it.

          Will come back to this if still available when time permits.

          Show
          Karthik Kambatla (Inactive) added a comment - Marking as unassigned, should anyone be interested in working on it. Will come back to this if still available when time permits.
          Hide
          Rohith added a comment -

          The issue is still exist in the latest code base. This issue require more discussion on whether to change the hard coded values which may break comatibility for the MR client OR bring out a new design to handle this dielema. If not planning to fix , I think this can be closed as wont fix.

          Show
          Rohith added a comment - The issue is still exist in the latest code base. This issue require more discussion on whether to change the hard coded values which may break comatibility for the MR client OR bring out a new design to handle this dielema. If not planning to fix , I think this can be closed as wont fix.

            People

            • Assignee:
              Unassigned
              Reporter:
              Nishan Shetty
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development