Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16119

Support "DROP TABLE ... PURGE" if Hive client supports it

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • SQL
    • None

    Description

      There's currently code that explicitly disables the "PURGE" flag when dropping a table:

          if (ctx.PURGE != null) {
            throw operationNotAllowed("DROP TABLE ... PURGE", ctx)
          }
      

      That flag is necessary in certain situations where the table data cannot be moved to the trash (which will be tried unless "PURGE" is requested). If the client supports it (Hive >= 0.14.0 according to the Hive docs), we should allow that option to be defined.

      For non-Hive tables, as far as I can understand, "PURGE" is the current behavior of Spark.

      The same limitation exists currently for "ALTER TABLE ... DROP PARTITION" so should probably also be covered by this change.

      Attachments

        Activity

          People

            vanzin Marcelo Masiero Vanzin
            vanzin Marcelo Masiero Vanzin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: