Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-42399

CONV() silently overflows returning wrong results

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      spark-sql> SELECT CONV(SUBSTRING('0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff', 3), 16, 10);

      18446744073709551615

      Time taken: 2.114 seconds, Fetched 1 row(s)

      spark-sql> set spark.sql.ansi.enabled = true;

      spark.sql.ansi.enabled true

      Time taken: 0.068 seconds, Fetched 1 row(s)

      spark-sql> SELECT CONV(SUBSTRING('0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff', 3), 16, 10);

      18446744073709551615

      Time taken: 0.05 seconds, Fetched 1 row(s)

      In ANSI mode we should raise an error for sure.
      In non ANSI either an error or a NULL maybe be acceptable.

      Alternatively, of course, we could consider if we can support arbitrary domains since the result is a STRING again. 

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            srielau Serge Rielau

            Dates

              Created:
              Updated:

              Slack

                Issue deployment