Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-30814

Add Columns references should be able to resolve each other

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      In ResolveAlterTableChanges, we have checks that make sure that positional arguments exist and are normalized around case sensitivity for ALTER TABLE ADD COLUMNS. However, we missed the case, where a column in ADD COLUMNS can depend on the position of a column that is just being added.

      For example for the schema:

      root:
        - a: string
        - b: long
       

       

      The following should work:

      ALTER TABLE ... ADD COLUMNS (x int AFTER a, y int AFTER x) 

      Currently, the above statement will throw an error saying that AFTER x cannot be resolved, because x doesn't exist yet.

      Attachments

        Activity

          People

            imback82 Terry Kim
            brkyvz Burak Yavuz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: