Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-38615 SQL Error Attribution Framework
  3. SPARK-38616

Keep track of SQL query text in Catalyst TreeNode

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.3.0
    • 3.3.0
    • SQL
    • None

    Description

      Spark SQL uses the class Origin for tracking the position of each TreeNode in the SQL query text. When there is a parser error, we can show the position info in the error message:

      > sql("create tabe foo(i int)")
      org.apache.spark.sql.catalyst.parser.ParseException:
      no viable alternative at input 'create tabe'(line 1, pos 7)
      
      
      == SQL ==
      create tabe foo(i int)
      -------^^^ 

      It contains two fields: line and startPosition. This is enough for the parser since the SQL query text is known.

      However, the SQL query text is unknown in the execution phase. One solution is to include the query text in Origin and show it in the runtime error message. 

      Attachments

        Activity

          People

            Gengliang.Wang Gengliang Wang
            Gengliang.Wang Gengliang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: