Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-431

HCatalogStore can't write any data using INSERT OVERWRITE clause.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.8.0
    • Component/s: Catalog
    • Labels:
      None

      Description

      If I write INSERT OVERWRITE INTO clause using HCatalogStore, HCatalogStore can't write any data even though scan table has data. I found this bug as follows :

      scan table
      tajo> select * from table1;
      Progress: 100%, response time: 1.402 sec
      final state: QUERY_SUCCEEDED, response time: 1.402 sec
      result: hdfs://localhost:9010/tmp/tajo-blrunner/staging/q_1387366394043_0002/RESULT, 5 rows (60 B)
      id,  name,  score,  type
      -------------------------------
      1,  ooo,  1.1,  a
      2,  ppp,  2.3,  b
      3,  qqq,  3.4,  c
      4,  rrr,  4.5,  d
      5,  xxx,  5.6,  e
      
      insert query
      tajo> insert overwrite into table3 select * from table1;
      Progress: 0%, response time: 1.975 sec
      Progress: 100%, response time: 3.69 sec
      final state: QUERY_SUCCEEDED, response time: 3.69 sec
      OK
      
      target table
      tajo> select * from table3;
      Progress: 100%, response time: 0.216 sec
      final state: QUERY_SUCCEEDED, response time: 0.216 sec
      result: hdfs://localhost:9010/tmp/tajo-blrunner/staging/q_1387366394043_0004/RESULT, 0 rows (0 B)
      id,  name,  score,  type
      -------------------------------
      tajo> 
      
      1. TAJO-431_2.patch
        6 kB
        Jaehwa Jung
      2. TAJO-431.patch
        3 kB
        Jaehwa Jung

        Activity

        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-trunk-postcommit #632 (See https://builds.apache.org/job/Tajo-trunk-postcommit/632/)
        TAJO-431: HCatalogStore can't write any data using INSERT OVERWRITE clause. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=302824a390fcbe0f51914dd0347c5a17da22d298)

        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/pom.xml
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java
        • CHANGES.txt
        • tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/TestHCatalogStore.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-trunk-postcommit #632 (See https://builds.apache.org/job/Tajo-trunk-postcommit/632/ ) TAJO-431 : HCatalogStore can't write any data using INSERT OVERWRITE clause. (jaehwa) (jhjung: https://git-wip-us.apache.org/repos/asf?p=incubator-tajo.git&a=commit&h=302824a390fcbe0f51914dd0347c5a17da22d298 ) tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/pom.xml tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/main/java/org/apache/tajo/catalog/store/HCatalogStore.java CHANGES.txt tajo-catalog/tajo-catalog-drivers/tajo-hcatalog/src/test/java/org/apache/tajo/catalog/store/TestHCatalogStore.java
        Hide
        blrunner Jaehwa Jung added a comment -

        Hyunsik, thank you for your review.
        I committed just now.

        Show
        blrunner Jaehwa Jung added a comment - Hyunsik, thank you for your review. I committed just now.
        Hide
        hyunsik Hyunsik Choi added a comment -

        +1
        The fix looks good for me.

        Show
        hyunsik Hyunsik Choi added a comment - +1 The fix looks good for me.
        Hide
        blrunner Jaehwa Jung added a comment -

        I uploaded the second patch.

        Show
        blrunner Jaehwa Jung added a comment - I uploaded the second patch.
        Hide
        blrunner Jaehwa Jung added a comment -

        HiveMetaStoreClient provided several types dropTable API. Existing HCatalogStore used dropTable API which deleted table schema and data files. But Tajo CatalogStore policy is not to delete data files. So, I modified another dropTable API at HCatalogStore.

        Review the patch, please.

        Show
        blrunner Jaehwa Jung added a comment - HiveMetaStoreClient provided several types dropTable API. Existing HCatalogStore used dropTable API which deleted table schema and data files. But Tajo CatalogStore policy is not to delete data files. So, I modified another dropTable API at HCatalogStore. Review the patch, please.

          People

          • Assignee:
            blrunner Jaehwa Jung
            Reporter:
            blrunner Jaehwa Jung
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development