Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-34735

Add modified configs for SQL execution in UI

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 3.2.0
    • 3.3.0
    • SQL
    • None

    Description

      For SQL user, it's very common to add some config to optimize sql. Within a script, it would look like this

      set k1=v1;
      set k2=v2;
      set ...
      
      INSERT INTO TABLE t1
      SELECT ...
      
      

       
      It's hard to find the configs used by sql without the raw sql string. Current UI provide a `Environment` tab that we can only get some global initial config, however it's not enough.

      Some use case:

      • Jar based job, we might set config many times due to many sql execution.
      • SQL server e.g. (SparkThriftServer), we might execute thousands scripts every day with different session.

      We expect a feature that can list the modified configs which could affect the sql execution.

       

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ulysses XiDuo You
            ulysses XiDuo You
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment