Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43063

`df.show` handle null should print NULL instead of null

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Trivial
    • Resolution: Fixed
    • 3.4.0
    • 3.5.0
    • SQL
    • None

    Description

      `df.show` handle null should print NULL instead of null to consistent behavior;

      Like as the following behavior is currently inconsistent:
      ``` shell
      scala> spark.sql("select decode(6, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattle') as result").show(false)
      +------+
      |result|
      +------+
      |null  |
      +------+
      ```
      ``` shell
      spark-sql> DESC FUNCTION EXTENDED decode;
      function_desc
      Function: decode
      Class: org.apache.spark.sql.catalyst.expressions.Decode
      Usage:
          decode(bin, charset) - Decodes the first argument using the second argument character set.
      
          decode(expr, search, result [, search, result ] ... [, default]) - Compares expr
            to each search value in order. If expr is equal to a search value, decode returns
            the corresponding result. If no match is found, then it returns default. If default
            is omitted, it returns null.
      
      Extended Usage:
          Examples:
            > SELECT decode(encode('abc', 'utf-8'), 'utf-8');
             abc
            > SELECT decode(2, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattle', 'Non domestic');
             San Francisco
            > SELECT decode(6, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattle', 'Non domestic');
             Non domestic
            > SELECT decode(6, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattle');
             NULL
      
          Since: 3.2.0
      
      Time taken: 0.074 seconds, Fetched 4 row(s)
      ```
      ``` shell
      spark-sql> select decode(6, 1, 'Southlake', 2, 'San Francisco', 3, 'New Jersey', 4, 'Seattle');
      NULL
      

      Attachments

        Activity

          People

            kaifeiYi yikaifei
            kaifeiYi yikaifei
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: