Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21598

Spark Thrift Server stopped after express upgrade due to undefined port

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.5.2
    • Fix Version/s: 2.5.2
    • Component/s: ambari-upgrade
    • Labels:
      None
    • Environment:

      Source Ambari Version:- 2.2.2
      Target Ambari Version:-:-ambari-2.5.2.0-189
      Source Stack:-BigInsights-4.2.0.0
      target Stack:- HDP-2.6.2.0-124

      Description

      Steps to reproduce:-
      1. Installed a IOP cluster ambari-version:- 2.2.0,BigInsights-4.2.0.0
      2. Upgrade the ambari from 2.2.0 to 2.5.2.0-189(IOP Clusters)
      3. Remove IOP Select.
      4. Register HDP Stack to HDP-2.6.2.0-124
      5. EU
      6. Post EU

      Spark Thrift Server is failing after Upgrade:-

      Logs:-

      17/07/28 03:32:18 INFO SparkUI: Stopped Spark web UI at http://natr66-tbus-iop420tofnsec-r6-4.openstacklocal:4040
      17/07/28 03:32:18 INFO YarnClientSchedulerBackend: Interrupting monitor thread
      17/07/28 03:32:18 INFO YarnClientSchedulerBackend: Shutting down all executors
      17/07/28 03:32:18 INFO YarnClientSchedulerBackend: Asking each executor to shut down
      17/07/28 03:32:18 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices
      (serviceOption=None,
       services=List(),
       started=false)
      17/07/28 03:32:18 INFO YarnClientSchedulerBackend: Stopped
      17/07/28 03:32:18 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
      17/07/28 03:32:18 INFO MemoryStore: MemoryStore cleared
      17/07/28 03:32:18 INFO BlockManager: BlockManager stopped
      17/07/28 03:32:18 INFO BlockManagerMaster: BlockManagerMaster stopped
      17/07/28 03:32:18 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
      17/07/28 03:32:18 INFO SparkContext: Successfully stopped SparkContext
      17/07/28 03:32:18 ERROR Utils: Uncaught exception in thread pool-7-thread-1
      java.lang.NullPointerException
              at org.apache.spark.sql.hive.thriftserver.HiveThriftServer2$$anonfun$main$1.apply$mcV$sp(HiveThriftServer2.scala:123)
              at org.apache.spark.util.SparkShutdownHook.run(ShutdownHookManager.scala:267)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(ShutdownHookManager.scala:239)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:239)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1$$anonfun$apply$mcV$sp$1.apply(ShutdownHookManager.scala:239)
              at org.apache.spark.util.Utils$.logUncaughtExceptions(Utils.scala:1817)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply$mcV$sp(ShutdownHookManager.scala:239)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:239)
              at org.apache.spark.util.SparkShutdownHookManager$$anonfun$runAll$1.apply(ShutdownHookManager.scala:239)
              at scala.util.Try$.apply(Try.scala:161)
              at org.apache.spark.util.SparkShutdownHookManager.runAll(ShutdownHookManager.scala:239)
              at org.apache.spark.util.SparkShutdownHookManager$$anon$2.run(ShutdownHookManager.scala:218)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      17/07/28 03:32:18 INFO ShutdownHookManager: Shutdown hook called
      17/07/28 03:32:18 INFO ShutdownHookManager: Deleting directory /tmp/spark-87997670-a290-4c52-a5f5-4ea0bbe87d4c
      17/07/28 03:32:18 INFO ShutdownHookManager: Deleting directory /tmp/spark-f76d3d61-f7d5-4a0a-a50d-d3a1766f3f09
      17/07/28 03:32:18 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
      17/07/28 03:32:18 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
      

        Attachments

        1. AMBARI-21598.patch
          3 kB
          Attila Doroszlai

          Issue Links

            Activity

              People

              • Assignee:
                adoroszlai Attila Doroszlai
                Reporter:
                pradarttana@gmail.com Pradarttana
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: