Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-39259

Timestamps returned by now() and equivalent functions are not consistent in subqueries

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.2.1
    • 3.3.0, 3.1.4, 3.2.2, 3.4.0
    • Optimizer

    • Reproduced in the Spark Shell on the current 3.4.0 snapshot

    Description

      Timestamp evaluation in not consistent across subqueries. As an example for the Spark Shell

       

      sql("SELECT * FROM (SELECT 1) WHERE now() IN (SELECT now())").collect()

      Returns an empty result.

       

      The root cause is that ComputeCurrentTime does not iterate into subqueries

      Attachments

        Activity

          People

            olaky Jan-Ole Sasse
            olaky Jan-Ole Sasse
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: