Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-31941

Handling the exception in SparkUI for getSparkUser method

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.6, 3.0.0, 3.1.0
    • Fix Version/s: 2.4.7, 3.0.1, 3.1.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      After SPARK-31632 SparkException is thrown from def applicationInfo(

        def applicationInfo(): v1.ApplicationInfo = {
          try {
            // The ApplicationInfo may not be available when Spark is starting up.
            store.view(classOf[ApplicationInfoWrapper]).max(1).iterator().next().info
          } catch {
            case _: NoSuchElementException =>
              throw new SparkException("Failed to get the application information. " +
                "If you are starting up Spark, please wait a while until it's ready.")
          }
        }
      

      Where as the caller for this method def getSparkUser in Spark UI is not handling SparkException in the catch

        def getSparkUser: String = {
          try {
            Option(store.applicationInfo().attempts.head.sparkUser)
              .orElse(store.environmentInfo().systemProperties.toMap.get("user.name"))
              .getOrElse("<unknown>")
          } catch {
            case _: NoSuchElementException => "<unknown>"
          }
        }
      

      So On using this method (getSparkUser )we can get the application erred out.

      So either we should thow

       throw new NoSuchElementException("Failed to get the application information. " +
                "If you are starting up Spark, please wait a while until it's ready.")
      

      or else add the scenario to catch spark exception in getSparkUser
      case _: SparkException => "<unknown>"

        Attachments

          Activity

            People

            • Assignee:
              saurabhc100 Saurabh Chawla
              Reporter:
              saurabhc100 Saurabh Chawla
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: