Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-50371

Introduce a shared plan representation

Rank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 4.0.0
    • None
    • Connect, SQL
    • None

    Description

      Introduce a stable structured plan representation that can be used by both Connect and Classic. This can be used by explain, queryExecution, or a lightweight version of QueryExecutionListener.

      This plan representation should support the following things:

      • (Parsed/Logical/Pysical)Plans
        • Schema
        • Name
        • Children
      • Expressions
        • Name
        • Children
      • Subqueries
      • Reuse of subqueries/plans.
      • Metrics
      • (optionally) Statistics

      The plan representation must be serializable to and from proto.

      For connect we may want to send it at the end of query execution. We do need to think about limits here.

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            hvanhovell Herman van Hövell
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment