Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-6314

The logging (progress reporting) is too verbose

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      The progress report is issued every second even when no progress have been made:

      2014-01-27 10:35:55,209 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 sec
      2014-01-27 10:35:56,678 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 sec
      2014-01-27 10:35:59,344 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 6.68 sec
      2014-01-27 10:36:01,268 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 8.67 sec
      2014-01-27 10:36:03,149 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 8.67 sec
      

      This pollutes the logs and the screen, and people do not appreciate it as much as the designers might have thought (How do I limit log verbosity of hive?, controlling the level of verbosity in Hive).
      It would be nice to be able to control the level of verbosity (but not by the -v switch!):

      1. Make sure that the progress report is only issued where there is something new to report; or
      2. Remove all the progress messages; or
      3. Make sure that progress is reported only every X sec (instead of every 1 second)

        Attachments

        1. HIVE-6314.1.patch.txt
          5 kB
          Navis Ryu
        2. HIVE-6314.2.patch
          5 kB
          Harish Butani

          Activity

            People

            • Assignee:
              navis Navis Ryu
              Reporter:
              sds Sam Steingold
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: