Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29112

Expose more details when ApplicationMaster reporter faces a fatal exception

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.4, 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: YARN
    • Labels:
      None

      Description

      In ApplicationMaster.Reporter thread, fatal exception information is swallowed. It's better to expose.
      A thrift server was shutdown due to some fatal exception but no useful information from log.

      19/09/16 06:59:54,498 INFO [Reporter] yarn.ApplicationMaster:54 : Final app status: FAILED, exitCode: 12, (reason: Exception was thrown 1 time(s) from Reporter thread.)
      19/09/16 06:59:54,500 ERROR [Driver] thriftserver.HiveThriftServer2:91 : Error starting HiveThriftServer2
      java.lang.InterruptedException: sleep interrupted
              at java.lang.Thread.sleep(Native Method)
              at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$.main(HiveThriftServer2.scala:160)
              at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.main(HiveThriftServer2.scala)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$4.run(ApplicationMaster.scala:708)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cltlfcjin Lantao Jin
                Reporter:
                cltlfcjin Lantao Jin
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: