Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-2678

Programmatically limit CLI status updates

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • CLI, Logging
    • None

    Description

      Provide a way to configure the frequency of Hive logging output, i.e., these:

      2011-12-23 22:31:20,979 Stage-1 map = 16%, reduce = 0%, Cumulative CPU 567.27 sec

      Some jobs update more than once per second, which is way more than necessary (and runs users out of scrollback buffer when using the CLI in screen).

      Default should be to update when map % or reduce % complete has gone up by one, and should be configurable via "SET mapred.update.rate=N;" to indicate that I would like updates every N seconds.

      Attachments

        Activity

          People

            Unassigned Unassigned
            akramer Adam Kramer
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: