Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12957

Derive and propagate data constrains in logical plan

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • SQL
    • None

    Description

      Based on the semantic of a query plan, we can derive data constrains (e.g. if a filter defines a > 10, we know that the output data of this filter satisfy the constrain of a > 10 and a is not null). We should build a framework to derive and propagate constrains in the logical plan, which can help us to build more advanced optimizations.

      Attachments

        1. ConstraintPropagationinSparkSQL.pdf
          199 kB
          Sameer Agarwal

        Issue Links

          Activity

            People

              sameerag Sameer Agarwal
              yhuai Yin Huai
              Votes:
              2 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: