Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36804

Using the verbose parameter in yarn mode would cause application submission failure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.1.2
    • 3.3.0
    • YARN
    • None

    Description

      If we submit the spark application with the --verbose parameter in yarn mode, we will get the following exception:

      Exception in thread "main" java.lang.IllegalArgumentException: Unknown/unsupported param List(--verbose)Exception in thread "main" 
      

      I check the code, found SparkSubmit invoke YarnClusterApplication with --verbose arguments, however ClientArguments used by YarnClusterApplication don't support that argument by now, as a result, IllegalArgumentException is thrown in ClientArguments. I think we can support --verbose in YARN mode to keep consistency with other module
       

      Attachments

        Activity

          People

            Li Jianmeng Jianmeng Li
            Li Jianmeng Jianmeng Li
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: