Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22454

ExternalShuffleClient.close() should check null

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.3.0
    • Component/s: Shuffle
    • Labels:
      None

      Description

      17/11/06 20:08:05 ERROR Utils: Uncaught exception in thread main
      java.lang.NullPointerException
      	at org.apache.spark.network.shuffle.ExternalShuffleClient.close(ExternalShuffleClient.java:152)
      	at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1407)
      	at org.apache.spark.SparkEnv.stop(SparkEnv.scala:89)
      	at org.apache.spark.SparkContext$$anonfun$stop$11.apply$mcV$sp(SparkContext.scala:1849)
      	at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1283)
      	at org.apache.spark.SparkContext.stop(SparkContext.scala:1848)
      	at org.apache.spark.SparkContext.<init>(SparkContext.scala:587)
      	at org.apache.spark.SparkContext$.getOrCreate(SparkContext.scala:2320)
      	at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:868)
      	at org.apache.spark.sql.SparkSession$Builder$$anonfun$6.apply(SparkSession.scala:860)
      	at scala.Option.getOrElse(Option.scala:121)
      	at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:860)
      	at org.apache.spark.sql.hive.thriftserver.SparkSQLEnv$.init(SparkSQLEnv.scala:47)
      	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.<init>(SparkSQLCLIDriver.scala:288)
      	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver$.main(SparkSQLCLIDriver.scala:137)
      	at org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver.main(SparkSQLCLIDriver.scala)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:743)
      	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:187)
      	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:212)
      	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:126)
      	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
      

        Attachments

          Activity

            People

            • Assignee:
              yumwang Yuming Wang
              Reporter:
              yumwang Yuming Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: