Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-37452

Char and Varchar breaks backward compatibility between v3 and v2

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.2, 3.2.0
    • 3.1.3, 3.2.1, 3.3.0
    • SQL
    • None

    Description

      We will store table schema in table properties for the read-side to restore. In Spark 3.1, we add char/varchar support natively. In some commands like `create table`, `alter table` with these types,  the char or varchar will be stored directly to those properties. If a user uses Spark 2 to read such a table it will fail to parse the schema.

      A table can be a newly created one by Spark 3.1 and later or an existing one modified by Spark 3.1 and on.  

      Attachments

        Activity

          People

            Qin Yao Kent Yao 2
            Qin Yao Kent Yao 2
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: