Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15541

SparkContext.stop throws error

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Resolved
    • Affects Version/s: None
    • Fix Version/s: 1.6.3, 2.0.1, 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      When running unit-tests or examples from command line or Intellij, SparkContext throws errors.
      For example:
      ./bin/run-example ml.JavaNaiveBayesExample
      Exception in thread "main" 16/05/25 15:17:55 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
      java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
      at org.apache.spark.rpc.netty.Dispatcher.stop(Dispatcher.scala:176)
      at org.apache.spark.rpc.netty.NettyRpcEnv.cleanup(NettyRpcEnv.scala:291)
      at org.apache.spark.rpc.netty.NettyRpcEnv.shutdown(NettyRpcEnv.scala:269)
      at org.apache.spark.SparkEnv.stop(SparkEnv.scala:91)
      at org.apache.spark.SparkContext$$anonfun$stop$11.apply$mcV$sp(SparkContext.scala:1796)
      at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1219)
      at org.apache.spark.SparkContext.stop(SparkContext.scala:1795)
      at org.apache.spark.sql.SparkSession.stop(SparkSession.scala:577)
      at org.apache.spark.examples.ml.JavaNaiveBayesExample.main(JavaNaiveBayesExample.java:61)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      16/05/25 15:17:55 INFO ShutdownHookManager: Shutdown hook called

        Attachments

          Activity

            People

            • Assignee:
              maver1ck Maciej BryƄski
              Reporter:
              wm624 Miao Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: