Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21902

BlockManager.doPut will hide actually exception when exception thrown in finally block

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.3.0
    • Component/s: Block Manager
    • Labels:
      None

      Description

      As logging below, actually exception will be hidden when removeBlockInternal throw an exception.

      2017-08-31,10:26:57,733 WARN org.apache.spark.storage.BlockManager: Putting block broadcast_110 failed due to an exception
      2017-08-31,10:26:57,734 WARN org.apache.spark.broadcast.BroadcastManager: Failed to create a new broadcast in 1 attempts
      java.io.IOException: Failed to create local dir in /tmp/blockmgr-5bb5ac1e-c494-434a-ab89-bd1808c6b9ed/2e.
              at org.apache.spark.storage.DiskBlockManager.getFile(DiskBlockManager.scala:70)
              at org.apache.spark.storage.DiskStore.remove(DiskStore.scala:115)
              at org.apache.spark.storage.BlockManager.removeBlockInternal(BlockManager.scala:1339)
              at org.apache.spark.storage.BlockManager.doPut(BlockManager.scala:910)
              at org.apache.spark.storage.BlockManager.doPutIterator(BlockManager.scala:948)
              at org.apache.spark.storage.BlockManager.putIterator(BlockManager.scala:726)
              at org.apache.spark.storage.BlockManager.putSingle(BlockManager.scala:1233)
              at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:122)
              at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88)
              at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
              at org.apache.spark.broadcast.BroadcastManager$$anonfun$newBroadcast$1.apply$mcVI$sp(BroadcastManager.scala:60)
              at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
              at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:58)
              at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1415)
              at org.apache.spark.scheduler.DAGScheduler.submitMissingTasks(DAGScheduler.scala:1002)
              at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$submitStage(DAGScheduler.scala:924)
              at org.apache.spark.scheduler.DAGScheduler$$anonfun$submitWaitingChildStages$6.apply(DAGScheduler.scala:771)
              at org.apache.spark.scheduler.DAGScheduler$$anonfun$submitWaitingChildStages$6.apply(DAGScheduler.scala:770)
              at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
              at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
              at org.apache.spark.scheduler.DAGScheduler.submitWaitingChildStages(DAGScheduler.scala:770)
              at org.apache.spark.scheduler.DAGScheduler.handleTaskCompletion(DAGScheduler.scala:1235)
              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1662)
              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1620)
              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1609)
              at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)
      

      I want to print the exception first for troubleshooting.Or may be we should not throw exception when removing blocks.

        Attachments

          Activity

            People

            • Assignee:
              cane zhoukang
              Reporter:
              cane zhoukang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: