Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-36588

Use v2 commands by default

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • 3.3.0
    • None
    • SQL
    • None

    Description

      It's been a while after we introduce the v2 commands, and I think it's time to use v2 commands by default even for the session catalog, with a legacy config to fall back to the v1 commands.

      We can do this one command by one command, with tests for both the v1 and v2 versions. The tests should help us understand the behavior difference between v1 and v2 commands, so that we can:

      1. fix the v2 commands to match the v1 behavior
      2. or accept the behavior difference and write migration guide

      We can reuse the test framework built in https://issues.apache.org/jira/browse/SPARK-33381

      Attachments

        1.
        		 Migrate SHOW TABLES to use V2 command by default Sub-task Resolved Terry Kim
        2.
        		 Migrate SHOW NAMESPACES to use V2 command by default Sub-task Resolved Terry Kim
        3.
        		 Migrate DESCRIBE NAMESPACE to use V2 command by default Sub-task Resolved Terry Kim
        4.
        		 Migrate SHOW TBLPROPERTIES to use V2 command by default Sub-task Resolved PengLei
        5.
        		 Migrate ALTER NAMESPACE ... SET PROPERTIES to use v2 command by default Sub-task Resolved Terry Kim
        6.
        		 Migrate ALTER NAMESPACE ... SET LOCATION to use v2 command by default Sub-task Resolved Terry Kim
        7.
        		 ALTER NAMESPACE ... SET LOCATION should handle empty location consistently across v1 and v2 command Sub-task Resolved Terry Kim
        8.
        		 CREATE NAMESPACE should qualify location for v2 command Sub-task Resolved Terry Kim
        9.
        		 Migrate SHOW CREATE TABLE to use V2 command by default Sub-task Open Unassigned
        10.
        		 Migrate DROP NAMESPACE to use V2 command by default Sub-task Resolved dch nguyen
        11.
        		 CREATE/REPLACE TABLE should qualify location for v2 command Sub-task Resolved PengLei
        12.
        		 Keep consistent order of columns with user specify for v1 table Sub-task In Progress Unassigned
        13.
        		 ALTER NAMESPACE ... SET PROPERTIES v2 command should use existing properties in addition to new properties Sub-task Closed Unassigned
        14.
        		 Migrate CREATE NAMESPACE to use v2 command by default Sub-task Resolved Terry Kim
        15.
        		 Migrate SHOW TABLES EXTENDED to use v2 command by default Sub-task Open Unassigned
        16.
        		 Put the some built-in table properties into V1Table.propertie to adapt to V2 command Sub-task Resolved PengLei
        17.
        		 Migrate SHOW CREATE TABLE to use v2 command by default Sub-task Resolved PengLei
        18.
        		 Migrate DESCRIBE TABLE to use V2 command by default Sub-task Open Terry Kim
        19.
        		 Support cascade mode for `dropNamespace` API Sub-task Resolved dch nguyen

        Activity

          People

            imback82 Terry Kim
            cloud_fan Wenchen Fan
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: