Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24590

Operation Logging still leaks the log4j Appenders

    XMLWordPrintableJSON

Details

    Description

      I'm using Hive 3.1.2 with options below.

      • hive.server2.logging.operation.enabled=true
      • hive.server2.logging.operation.level=VERBOSE
      • hive.async.log.enabled=false

      I already know the ticket, https://issues.apache.org/jira/browse/HIVE-17128 but HS2 still leaks log4j RandomAccessFileManager.

      I checked the operation log file which is not closed/deleted properly.

      Then there's the log,

      client.TezClient: Shutting down Tez Session, sessionName= ....

      Attachments

        1. Screen Shot 2021-01-06 at 18.42.24.png
          231 kB
          Eugene Chung
        2. Screen Shot 2021-01-06 at 18.42.55.png
          86 kB
          Eugene Chung
        3. Screen Shot 2021-01-06 at 18.42.05.png
          289 kB
          Eugene Chung
        4. add_debug_log_and_trace.patch
          2 kB
          Eugene Chung
        5. Screen Shot 2021-01-06 at 21.38.32.png
          185 kB
          Eugene Chung
        6. Screen Shot 2021-01-06 at 21.47.28.png
          92 kB
          Eugene Chung
        7. Screen Shot 2021-01-08 at 21.01.40.png
          62 kB
          Eugene Chung

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. HIVE-25970 Missing messages in HS2 operation logs

          • Major - Major loss of function.
          • Closed
          is a child of

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-22753 Fix gradual mem leak: Operationlog related appenders should be cleared up on errors

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-26014 Remove redundant HushableRandomAccessFileAppender

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-17128 Operation Logging leaks file descriptors as the log4j Appender is never closed

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-24569 LLAP daemon leaks file descriptors/log4j appenders

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #1849

          Web Link GitHub Pull Request #2432

          Activity

            People

              zabetak Stamatis Zampetakis
              euigeun_chung Eugene Chung
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6h 20m
                  6h 20m