[HIVE-16675] Fix ConcurrentModificationException in SparkClientImpl#startDriver

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Assignee: liyunzhang
• Reporter: liyunzhang
• Created: 16/May/17 08:27
• Updated: 22/May/18 23:57
• Resolved: 25/May/17 06:03

Description

the exception is

              2017-05-16T00:29:37,480  WARN [Driver] client.SparkClientImpl: Exception while waiting for child process.
              3926 java.util.ConcurrentModificationException
              3927         at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) ~[?:1.8.0_91]
              3928         at java.util.ArrayList$Itr.next(ArrayList.java:851) ~[?:1.8.0_91]
              3929         at org.apache.hive.spark.client.SparkClientImpl$3.run(SparkClientImpl.java:495) [hive-exec-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
              3930         at java.lang.Thread.run(Thread.java:745) [?:1.8.0_91]

It seems that SparkClientImpl.java#childErrorLog is read while it is written. It is better to change SparkClientImpl.java#childErrorLog from ArrayList to CopyOnWriteArrayList to avoid the exception.