Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.5.0
    • 3.5.0
    • PySpark
    • None

    Description

      Support UDTFs with empty returns, for example:

      @udtf(returnType="a: int")
      class TestUDTF:
          def eval(self, a: int):
              ... 

      Currently, this will fail with the exception 

      TypeError: 'NoneType' object is not iterable 

      Another example

      class TestUDTF:
          def eval(self, a: int):
              yield 

      This will fail with the exceptions

      java.lang.NullPointerException 

      Note, arrow-optimized UDTFs already support this. This error only occurs with regular Python UDTFs.

      Attachments

        Activity

          gurwls223 Hyukjin Kwon added a comment - Fixed in https://github.com/apache/spark/pull/42044

          People

            allisonwang-db Allison Wang
            allisonwang-db Allison Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: