Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17861 Store data source partitions in metastore and push partition pruning into metastore
  3. SPARK-18183

INSERT OVERWRITE TABLE ... PARTITION will overwrite the entire Datasource table instead of just the specified partition

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0
    • SQL
    • None

    Description

      In Hive, this will only overwrite the specified partition. We should fix this for Datasource tables to be more in line with the Hive behavior.

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue SPARK-18184 INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables cannot handle partitions with custom locations

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue SPARK-18185 Should fix INSERT OVERWRITE TABLE of Datasource tables with dynamic partitions

          • Major - Major loss of function.
          • Resolved
          links to

          Pull request #15705 [Github] Pull Request #15705 (ericl)

          Activity

            People

              ekhliang Eric Liang
              ekhliang Eric Liang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: