Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18384

ConcurrentModificationException in log4j2.x library

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: Logging
    • Labels:
      None
    • Target Version/s:

      Description

      In one of the internal testing, observed the following exception

      java.util.ConcurrentModificationException
      	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909) ~[?:1.8.0_152]
      	at java.util.ArrayList$Itr.next(ArrayList.java:859) ~[?:1.8.0_152]
      	at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1042) ~[?:1.8.0_152]
      	at org.apache.logging.log4j.message.ParameterFormatter.appendCollection(ParameterFormatter.java:596) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.message.ParameterFormatter.appendPotentiallyRecursiveValue(ParameterFormatter.java:504) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.message.ParameterFormatter.recursiveDeepToString(ParameterFormatter.java:429) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.message.ParameterFormatter.formatMessage2(ParameterFormatter.java:189) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.message.ParameterizedMessage.formatTo(ParameterizedMessage.java:224) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.message.ParameterizedMessage.getFormattedMessage(ParameterizedMessage.java:200) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.RingBufferLogEvent.setMessage(RingBufferLogEvent.java:126) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.RingBufferLogEvent.setValues(RingBufferLogEvent.java:104) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.RingBufferLogEventTranslator.translateTo(RingBufferLogEventTranslator.java:56) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.RingBufferLogEventTranslator.translateTo(RingBufferLogEventTranslator.java:34) ~[log4j-core-2.6.2.jar:2.6.2]
      	at com.lmax.disruptor.RingBuffer.translateAndPublish(RingBuffer.java:930) ~[disruptor-3.3.0.jar:?]
      	at com.lmax.disruptor.RingBuffer.tryPublishEvent(RingBuffer.java:456) ~[disruptor-3.3.0.jar:?]
      	at org.apache.logging.log4j.core.async.AsyncLoggerDisruptor.tryPublish(AsyncLoggerDisruptor.java:190) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.AsyncLogger.publish(AsyncLogger.java:160) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.AsyncLogger.logWithThreadLocalTranslator(AsyncLogger.java:156) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.core.async.AsyncLogger.logMessage(AsyncLogger.java:126) ~[log4j-core-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2011) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1884) ~[log4j-api-2.6.2.jar:2.6.2]
      	at org.apache.logging.slf4j.Log4jLogger.info(Log4jLogger.java:189) ~[log4j-slf4j-impl-2.6.2.jar:2.6.2]
      	at org.apache.hadoop.hive.druid.security.KerberosHttpClient.inner_go(KerberosHttpClient.java:96) ~[hive-druid-handler-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
      	at org.apache.hadoop.hive.druid.security.KerberosHttpClient.access$100(KerberosHttpClient.java:50) ~[hive-druid-handler-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
      	at org.apache.hadoop.hive.druid.security.KerberosHttpClient$2.onSuccess(KerberosHttpClient.java:144) ~[hive-druid-handler-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
      	at org.apache.hadoop.hive.druid.security.KerberosHttpClient$2.onSuccess(KerberosHttpClient.java:134) ~[hive-druid-handler-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
      	at org.apache.hive.druid.com.google.common.util.concurrent.Futures$4.run(Futures.java:1181) ~[hive-druid-handler-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_152]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_152]
      	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
      

      The fix for this went into 2.9.1 LOG4J2-1988 onwards. Updating log4j to latest version should have a fix for this issue.

        Attachments

        1. HIVE-18384.1.patch
          1 kB
          Prasanth Jayachandran
        2. HIVE-18384.2.patch
          2 kB
          Prasanth Jayachandran
        3. HIVE-18384.3.patch
          2 kB
          Prasanth Jayachandran

          Activity

            People

            • Assignee:
              prasanth_j Prasanth Jayachandran
              Reporter:
              prasanth_j Prasanth Jayachandran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: