Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-304

drop table command should not remove data files in default

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: Catalog, QueryMaster
    • Labels:
      None

      Description

      Problem

      In the current implementation, Tajo removes a data directory when a user issues 'drop table' command. But, it is very dangerous in many cases. For example, some users may lost large data sets.

      Solution

      In default, 'DROP TABLE' should not remove a data directory. , and we need to add a config property 'tajo.command.drop_table.data_removal' for those who want to change the behavior of 'drop table'. In addition, Tajo should provide 'DROP TABLE table_name PURGE' for removing all data.

      1. TAJO-304.patch
        24 kB
        Hyunsik Choi

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        I changed the plan. I modified DDL statement as follows:

        DROP TABLE table_name [PURGE]
        

        PURGE is given, 'DROP TABLE' deletes the table from catalog and purges all data. Otherwise, 'DROP TABLE' just deletes a table from catalog.

        Show
        hyunsik Hyunsik Choi added a comment - I changed the plan. I modified DDL statement as follows: DROP TABLE table_name [PURGE] PURGE is given, 'DROP TABLE' deletes the table from catalog and purges all data. Otherwise, 'DROP TABLE' just deletes a table from catalog.
        Hide
        jhkim Jinho Kim added a comment -

        +1
        Looks great for me. I've verified tajo cli

        Show
        jhkim Jinho Kim added a comment - +1 Looks great for me. I've verified tajo cli
        Hide
        jihoonson Jihoon Son added a comment -

        +1.
        Looks good to me, too.

        Show
        jihoonson Jihoon Son added a comment - +1. Looks good to me, too.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I've just committed the patch, fixing it as resolved. Thank you for the fast reviews.

        Show
        hyunsik Hyunsik Choi added a comment - I've just committed the patch, fixing it as resolved. Thank you for the fast reviews.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-trunk-postcommit #591 (See https://builds.apache.org/job/Tajo-trunk-postcommit/591/)
        TAJO-304: drop table command should not remove data files in default. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=c033d85cf065b9030e97e0b4c3596adacbbc255c)

        • tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/client/TestTajoClient.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/HiveConverter.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/DropTableNode.java
        • CHANGES.txt
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java
        • tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanner.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java
        • tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto
        • tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto
        • tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLLexer.g4
        • tajo-algebra/src/main/java/org/apache/tajo/algebra/DropTable.java
        • tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-trunk-postcommit #591 (See https://builds.apache.org/job/Tajo-trunk-postcommit/591/ ) TAJO-304 : drop table command should not remove data files in default. (hyunsik) (hyunsik: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=c033d85cf065b9030e97e0b4c3596adacbbc255c ) tajo-core/tajo-core-backend/src/test/java/org/apache/tajo/client/TestTajoClient.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/HiveConverter.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/cli/TajoCli.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/logical/DropTableNode.java CHANGES.txt tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4 tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/engine/planner/LogicalPlanner.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/master/GlobalEngine.java tajo-core/tajo-core-backend/src/main/proto/ClientProtos.proto tajo-core/tajo-core-backend/src/main/proto/TajoMasterClientProtocol.proto tajo-core/tajo-core-backend/src/main/antlr4/org/apache/tajo/engine/parser/SQLLexer.g4 tajo-algebra/src/main/java/org/apache/tajo/algebra/DropTable.java tajo-core/tajo-core-backend/src/main/java/org/apache/tajo/client/TajoClient.java

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development