Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16907

"INSERT INTO" overwrite old data when destination table encapsulated by backquote

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.1.0, 2.1.1
    • Fix Version/s: 4.0.0
    • Component/s: Parser
    • Labels:
      None

      Description

      A way to reproduce:

      create database tdb;
      use tdb;
      create table t1(id int);
      create table t2(id int);
      explain insert into `tdb.t1` select * from t2;
      
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      |                                                                      Explain                                                                      |
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      | STAGE DEPENDENCIES:                                                                                                                               |
      |   Stage-1 is a root stage                                                                                                                         |
      |   Stage-6 depends on stages: Stage-1 , consists of Stage-3, Stage-2, Stage-4                                                                      |
      |   Stage-3                                                                                                                                         |
      |   Stage-0 depends on stages: Stage-3, Stage-2, Stage-5                                                                                            |
      |   Stage-2                                                                                                                                         |
      |   Stage-4                                                                                                                                         |
      |   Stage-5 depends on stages: Stage-4                                                                                                              |
      |                                                                                                                                                   |
      | STAGE PLANS:                                                                                                                                      |
      |   Stage: Stage-1                                                                                                                                  |
      |     Map Reduce                                                                                                                                    |
      |       Map Operator Tree:                                                                                                                          |
      |           TableScan                                                                                                                               |
      |             alias: t2                                                                                                                             |
      |             Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                             |
      |             Select Operator                                                                                                                       |
      |               expressions: id (type: int)                                                                                                         |
      |               outputColumnNames: _col0                                                                                                            |
      |               Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                           |
      |               File Output Operator                                                                                                                |
      |                 compressed: false                                                                                                                 |
      |                 Statistics: Num rows: 0 Data size: 0 Basic stats: NONE Column stats: NONE                                                         |
      |                 table:                                                                                                                            |
      |                     input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                  |
      |                     output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                |
      |                     serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                   |
      |                     name: tdb.t1                                                                                                                  |
      |                                                                                                                                                   |
      |   Stage: Stage-6                                                                                                                                  |
      |     Conditional Operator                                                                                                                          |
      |                                                                                                                                                   |
      |   Stage: Stage-3                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       files:                                                                                                                                      |
      |           hdfs directory: true                                                                                                                    |
      |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
      |                                                                                                                                                   |
      |   Stage: Stage-0                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       tables:                                                                                                                                     |
      |           replace: true                                                                                                                           |
      |           table:                                                                                                                                  |
      |               input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                        |
      |               output format: org.apache.hadoop.hive.ql.io.RCFileOutputFormat                                                                      |
      |               serde: org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe                                                                         |
      |               name: tdb.t1                                                                                                                        |
      |                                                                                                                                                   |
      |   Stage: Stage-2                                                                                                                                  |
      |     Merge File Operator                                                                                                                           |
      |       Map Operator Tree:                                                                                                                          |
      |           RCFile Merge Operator                                                                                                                   |
      |       merge level: block                                                                                                                          |
      |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
      |                                                                                                                                                   |
      |   Stage: Stage-4                                                                                                                                  |
      |     Merge File Operator                                                                                                                           |
      |       Map Operator Tree:                                                                                                                          |
      |           RCFile Merge Operator                                                                                                                   |
      |       merge level: block                                                                                                                          |
      |       input format: org.apache.hadoop.hive.ql.io.RCFileInputFormat                                                                                |
      |                                                                                                                                                   |
      |   Stage: Stage-5                                                                                                                                  |
      |     Move Operator                                                                                                                                 |
      |       files:                                                                                                                                      |
      |           hdfs directory: true                                                                                                                    |
      |           destination: hdfs://hacluster/user/hive/warehouse/tdb.db/t1/.hive-staging_hive_2017-06-15_15-52-34_017_849305017872068583-1/-ext-10000  |
      |                                                                                                                                                   |
      +---------------------------------------------------------------------------------------------------------------------------------------------------+
      
      

      Note that 'replace: true' in move operator

        Attachments

        1. HIVE-16907.1.patch
          2 kB
          Bing Li
        2. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        3. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        4. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        5. HIVE-16907.03.patch
          20 kB
          Zoltan Haindrich
        6. HIVE-16907.02.patch
          18 kB
          Zoltan Haindrich

          Issue Links

            Activity

              People

              • Assignee:
                kgyrtkirk Zoltan Haindrich
                Reporter:
                nemon Nemon Lou
              • Votes:
                1 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: