Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21133

HighlyCompressedMapStatus#writeExternal throws NPE

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 2.2.0
    • 2.2.0
    • Spark Core
    • None

    Description

      Reproduce, set set spark.sql.shuffle.partitions>2000 with shuffle, for simple:

      spark-sql --executor-memory 12g --driver-memory 8g --executor-cores 7   -e "
        set spark.sql.shuffle.partitions=2001;
        drop table if exists spark_hcms_npe;
        create table spark_hcms_npe as select id, count(*) from big_table group by id;
      "
      

      Error logs:

      17/06/18 15:00:27 ERROR Utils: Exception encountered
      java.lang.NullPointerException
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply$mcV$sp(MapStatus.scala:171)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1303)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus.writeExternal(MapStatus.scala:167)
              at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459)
              at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
              at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
              at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply$mcV$sp(MapOutputTracker.scala:617)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1337)
              at org.apache.spark.MapOutputTracker$.serializeMapStatuses(MapOutputTracker.scala:619)
              at org.apache.spark.MapOutputTrackerMaster.getSerializedMapOutputStatuses(MapOutputTracker.scala:562)
              at org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:351)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      17/06/18 15:00:27 ERROR MapOutputTrackerMaster: java.lang.NullPointerException
      java.io.IOException: java.lang.NullPointerException
              at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1310)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus.writeExternal(MapStatus.scala:167)
              at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459)
              at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
              at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
              at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply$mcV$sp(MapOutputTracker.scala:617)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1337)
              at org.apache.spark.MapOutputTracker$.serializeMapStatuses(MapOutputTracker.scala:619)
              at org.apache.spark.MapOutputTrackerMaster.getSerializedMapOutputStatuses(MapOutputTracker.scala:562)
              at org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:351)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.NullPointerException
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply$mcV$sp(MapStatus.scala:171)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1303)
              ... 17 more
      17/06/18 15:00:27 INFO MapOutputTrackerMasterEndpoint: Asked to send map output locations for shuffle 0 to 10.17.47.20:50188
      17/06/18 15:00:27 ERROR Utils: Exception encountered
      java.lang.NullPointerException
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply$mcV$sp(MapStatus.scala:171)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus$$anonfun$writeExternal$2.apply(MapStatus.scala:167)
              at org.apache.spark.util.Utils$.tryOrIOException(Utils.scala:1303)
              at org.apache.spark.scheduler.HighlyCompressedMapStatus.writeExternal(MapStatus.scala:167)
              at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1459)
              at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1430)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
              at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378)
              at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
              at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply$mcV$sp(MapOutputTracker.scala:617)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.MapOutputTracker$$anonfun$serializeMapStatuses$1.apply(MapOutputTracker.scala:616)
              at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1337)
              at org.apache.spark.MapOutputTracker$.serializeMapStatuses(MapOutputTracker.scala:619)
              at org.apache.spark.MapOutputTrackerMaster.getSerializedMapOutputStatuses(MapOutputTracker.scala:562)
              at org.apache.spark.MapOutputTrackerMaster$MessageLoop.run(MapOutputTracker.scala:351)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      
      

      Attachments

        Activity

          People

            yumwang Yuming Wang
            yumwang Yuming Wang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: