Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19842 Informational Referential Integrity Constraints Support in Spark
  3. SPARK-23755

[Performance] Distinct elimination

Rank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsAdd voteVotersWatch issueWatchersConvert to IssueLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.1.0
    • None
    • SQL
    • None

    Description

      The Distinct requirement can be removed if it is proved that the operation produces unique output.

      select distinct d.deptno /* PK */, e.empname
from employee e, department d
where e.empno = d.mgrno /*PK = FK*/

      Internal query after rewrite:

      select d.deptno, e.empname
from employee e, department d
where e.empno = d.mgrno

      Attachments

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            ioana-delaney Ioana Delaney
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment