Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11438 Allow users to define nondeterministic UDFs
  3. SPARK-11469

Initial implementation of allowing users to define UDFs that will not be evaluated more than once

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • SQL
    • None

    Description

      Right now, we have a projection collapsing optimization rule, that will collapse multiple project operators by evaluating an expression multiple times. However, there are cases that we do not want to do it (e.g. for rand). This work is to let users define such UDF functions.

      Attachments

        Activity

          People

            Unassigned Unassigned
            yhuai Yin Huai
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: