Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16675

Fix ConcurrentModificationException in SparkClientImpl#startDriver

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      the exception is

                    2017-05-16T00:29:37,480  WARN [Driver] client.SparkClientImpl: Exception while waiting for child process.
                    3926 java.util.ConcurrentModificationException
                    3927         at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) ~[?:1.8.0_91]
                    3928         at java.util.ArrayList$Itr.next(ArrayList.java:851) ~[?:1.8.0_91]
                    3929         at org.apache.hive.spark.client.SparkClientImpl$3.run(SparkClientImpl.java:495) [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
                    3930         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]
      

      It seems that SparkClientImpl.java#childErrorLog is read while it is written. It is better to change SparkClientImpl.java#childErrorLog from ArrayList to CopyOnWriteArrayList to avoid the exception.

        Attachments

        1. HIVE-16675.1.patch
          2 kB
          liyunzhang
        2. HIVE-16675.patch
          1 kB
          liyunzhang

          Activity

            People

            • Assignee:
              kellyzly liyunzhang
              Reporter:
              kellyzly liyunzhang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: