Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11085

Excessive logging by org.apache.hadoop.util.Progress when value is NaN

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      MAPREDUCE-5671 fixed the "illegal" progress values that do not fall into (0,1) interval when the progress value is given. Whenever illegal value was encountered, LOG.warn would log that incident.

      As a result, each of the task's syslog will be full of WARN [main] org.apache.hadoop.util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0

      Each input record will contribute to one line of such log, leading to most
      of the tasks' syslog > 1GB.

      We will need to change the log level to debug to avoid such excessive logging.

        Attachments

          Activity

            People

            • Assignee:
              mitdesai Mit Desai
              Reporter:
              mitdesai Mit Desai
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: