Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15541

SparkContext.stop throws error

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Resolved
    • None
    • 1.6.3, 2.0.1, 2.1.0
    • None
    • None

    Description

      When running unit-tests or examples from command line or Intellij, SparkContext throws errors.
      For example:
      ./bin/run-example ml.JavaNaiveBayesExample
      Exception in thread "main" 16/05/25 15:17:55 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
      java.lang.NoSuchMethodError: java.util.concurrent.ConcurrentHashMap.keySet()Ljava/util/concurrent/ConcurrentHashMap$KeySetView;
      at org.apache.spark.rpc.netty.Dispatcher.stop(Dispatcher.scala:176)
      at org.apache.spark.rpc.netty.NettyRpcEnv.cleanup(NettyRpcEnv.scala:291)
      at org.apache.spark.rpc.netty.NettyRpcEnv.shutdown(NettyRpcEnv.scala:269)
      at org.apache.spark.SparkEnv.stop(SparkEnv.scala:91)
      at org.apache.spark.SparkContext$$anonfun$stop$11.apply$mcV$sp(SparkContext.scala:1796)
      at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1219)
      at org.apache.spark.SparkContext.stop(SparkContext.scala:1795)
      at org.apache.spark.sql.SparkSession.stop(SparkSession.scala:577)
      at org.apache.spark.examples.ml.JavaNaiveBayesExample.main(JavaNaiveBayesExample.java:61)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:724)
      at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:180)
      at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:205)
      at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:119)
      at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      16/05/25 15:17:55 INFO ShutdownHookManager: Shutdown hook called

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            maver1ck Maciej Bryński
            wm624 Miao Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment