Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
The progress report is issued every second even when no progress have been made:
2014-01-27 10:35:55,209 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68 sec 2014-01-27 10:35:56,678 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68 sec 2014-01-27 10:35:59,344 Stage-1 map = 100%, reduce = 0%, Cumulative CPU 6.68 sec 2014-01-27 10:36:01,268 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67 sec 2014-01-27 10:36:03,149 Stage-1 map = 100%, reduce = 100%, Cumulative CPU 8.67 sec
This pollutes the logs and the screen, and people do not appreciate it as much as the designers might have thought (How do I limit log verbosity of hive?, controlling the level of verbosity in Hive).
It would be nice to be able to control the level of verbosity (but not by the -v switch!):
- Make sure that the progress report is only issued where there is something new to report; or
- Remove all the progress messages; or
- Make sure that progress is reported only every X sec (instead of every 1 second)