Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43797 Python User-defined Table Functions
  3. SPARK-48966

Improve error message with invalid unresolved column reference in UDTF call

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.0.0
    • 4.0.0
    • SQL

    Description

      This bug covers improving an error message in the event of invalid UDTF calls. For example:

      select * from udtf(
        observed => TABLE(select column from t),
        value_col => classic_dollars
      )

      Currently we get:

      Unsupported subquery expression: Table arguments are used in a function where they are not supported:
      'UnresolvedTableValuedFunction [udtf], observed => table-argument#68918 [], value_col => 'classic_dollars, false
         +- Project ...
             +- SubqueryAlias ...
                +- Relation ...

      But the real error is that the user passed column identifier classic_dollars rather than string "classic_dollars" into the string argument.

      The core reason is that CheckAnalysis checks the analyzer output tree for unresolved nodes at the end rather than reporting the error at the time of attempted resolving of the corresponding expression or operator (in this case, the unresolved attribute reference resulting from the unquoted classic_dollars).

      Attachments

        Issue Links

          Activity

            People

              dtenedor Daniel
              dtenedor Daniel
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: