Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
2.6.0
-
None
-
None
-
Reviewed
Description
MAPREDUCE-5671 fixed the "illegal" progress values that do not fall into (0,1) interval when the progress value is given. Whenever illegal value was encountered, LOG.warn would log that incident.
As a result, each of the task's syslog will be full of WARN [main] org.apache.hadoop.util.Progress: Illegal progress value found, progress is Float.NaN. Progress will be changed to 0
Each input record will contribute to one line of such log, leading to most
of the tasks' syslog > 1GB.
We will need to change the log level to debug to avoid such excessive logging.