Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8599

Improve non-deterministic expression handling

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.4.0
    • 1.5.0
    • SQL
    • None

    Description

      Right now, we are using expressions for Random distribution generating expressions. But, we have to track them in lots of places in the optimizer to handle them carefully. Otherwise, these expressions will be treated as stateless expressions and have unexpected behaviors (e.g. SPARK-8023).

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              cloud_fan Wenchen Fan
              yhuai Yin Huai
              Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: