Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21170

Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.1
    • Fix Version/s: 2.2.1, 2.3.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      17/06/20 22:49:39 ERROR Executor: Exception in task 225.0 in stage 1.0 (TID 27225)
      java.lang.IllegalArgumentException: Self-suppression not permitted
              at java.lang.Throwable.addSuppressed(Throwable.java:1043)
              at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1400)
              at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1145)
              at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1125)
              at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
              at org.apache.spark.scheduler.Task.run(Task.scala:108)
              at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:341)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      
      17/06/20 22:52:32 INFO scheduler.TaskSetManager: Lost task 427.0 in stage 1.0 (TID 27427) on 192.168.1.121, executor 12: java.lang.IllegalArgumentException (Self-suppression not permitted) [duplicate 1]
      17/06/20 22:52:33 INFO scheduler.TaskSetManager: Starting task 427.1 in stage 1.0 (TID 27764, 192.168.1.122, executor 106, partition 427, PROCESS_LOCAL, 4625 bytes)
      17/06/20 22:52:33 INFO scheduler.TaskSetManager: Lost task 186.0 in stage 1.0 (TID 27186) on 192.168.1.122, executor 106: java.lang.IllegalArgumentException (Self-suppression not permitted) [duplicate 2]
      17/06/20 22:52:38 INFO scheduler.TaskSetManager: Starting task 186.1 in stage 1.0 (TID 27765, 192.168.1.121, executor 9, partition 186, PROCESS_LOCAL, 4625 bytes)
      17/06/20 22:52:38 WARN scheduler.TaskSetManager: Lost task 392.0 in stage 1.0 (TID 27392, 192.168.1.121, executor 9): java.lang.IllegalArgumentException: Self-suppression not permitted
      	at java.lang.Throwable.addSuppressed(Throwable.java:1043)
      	at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1400)
      	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1145)
      	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1125)
      	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
      	at org.apache.spark.scheduler.Task.run(Task.scala:108)
      	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:341)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Here it is trying to suppress the same Throwable instance and causing to throw the IllegalArgumentException which masks the original exception.

      I think it should not add to the suppressed if it is the same instance.

        Attachments

          Activity

            People

            • Assignee:
              devaraj.k Devaraj K
              Reporter:
              devaraj.k Devaraj K
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: