Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4952

Handle ConcurrentModificationExceptions in SparkEnv.environmentDetails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.2, 1.2.1, 1.3.0
    • Component/s: Spark Core, YARN
    • Labels:
      None

      Description

      the log:

      Exception in thread "main" 14/12/24 12:00:25 INFO cluster.YarnClientSchedulerBackend: Add WebUI Filter. org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter, Map(PROXY_HOSTS -> tuan200, PROXY_URI_BASES -> http://host:9082/proxy/application_1414231702825_488625), /proxy/application_1414231702825_488625
      java.util.ConcurrentModificationException
              at java.util.Hashtable$Enumerator.next(Hashtable.java:1167)
              at scala.collection.convert.Wrappers$JPropertiesWrapper$$anon$3.next(Wrappers.scala:458)
              at scala.collection.convert.Wrappers$JPropertiesWrapper$$anon$3.next(Wrappers.scala:454)
              at scala.collection.Iterator$class.toStream(Iterator.scala:1143)
              at scala.collection.AbstractIterator.toStream(Iterator.scala:1157)
              at scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1143)
              at scala.collection.Iterator$$anonfun$toStream$1.apply(Iterator.scala:1143)
              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1085)
              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1077)
              at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:1149)
              at scala.collection.immutable.Stream$$anonfun$filteredTail$1.apply(Stream.scala:1149)
              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1085)
              at scala.collection.immutable.Stream$Cons.tail(Stream.scala:1077)
              at scala.collection.immutable.Stream.length(Stream.scala:284)
              at scala.collection.SeqLike$class.sorted(SeqLike.scala:608)
              at scala.collection.AbstractSeq.sorted(Seq.scala:40)
              at org.apache.spark.SparkEnv$.environmentDetails(SparkEnv.scala:324)
              at org.apache.spark.SparkContext.postEnvironmentUpdate(SparkContext.scala:1319)
              at org.apache.spark.SparkContext.<init>(SparkContext.scala:344)
              at com.zhe800.toona.als.computation.DealCF$.main(DealCF.scala:497)
              at com.zhe800.toona.als.computation.DealCF.main(DealCF.scala)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.spark.deploy.SparkSubmit$.launch(SparkSubmit.scala:329)
              at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:75)
              at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      14/12/24 12:00:25 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gq Guoqiang Li
                Reporter:
                gq Guoqiang Li
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: