Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30375 Feature Parity between PostgreSQL and Spark (implementation-dependent behaviours)
  3. SPARK-28007

Caret operator (^) means bitwise XOR in Spark/Hive and exponentiation in Postgres/Redshift

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0
    • None
    • SQL
    • None

    Description

      The expression expr1 ^ expr2 has different meanings in Spark and Postgres:

      • In Postgres and Redshift , this returns expr1 raised to the exponent expr2 (additionally, the Postgres docs explicitly state that this operation is left-associative).
      • In Spark and Hive, this returns the bitwise exclusive OR of expr1 and expr2.

      I'm reporting this under the Postgres compatibility umbrella. If we have SQL dialect support (e.g. a Postgres compatibility dialect), maybe this behavior could be flagged there? Alternatively, maybe we could document this in a compatibility / porting guide?

      Attachments

        Issue Links

          relates to

          Sub-task - The sub-task of the issue SPARK-28027 Missing some mathematical operators

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              joshrosen Josh Rosen
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: