Hive
  1. Hive
  2. HIVE-6352

Hive 0.11 & Hive 0.12 - derby.log and TempStatsStore are created when using MySQL Metastore

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not a Problem
    • Affects Version/s: 0.12.0
    • Fix Version/s: None
    • Component/s: Metastore
    • Labels:
      None
    • Environment:

      hadoop 1.2.1

      hive 0.12.0

      Description

      Hi,

      I am facing this situation for the below mentioned hql file -

      > Hive MySQL Metastore is used for table t1

      > Hive MySQL Metastore is not used for table t2 (derby.log is created in the directory)

      ----------

      – database pp_test_hive_metastore

      drop database if exists pp_test_hive_metastore cascade;

      create database pp_test_hive_metastore;

      use pp_test_hive_metastore;

      – table t1

      create table t1 ( id int, name string );

      LOAD DATA LOCAL INPATH 'testHiveMetastore.txt' OVERWRITE INTO TABLE t1;

      select * from t1;

      select count from t1;

      – table t2

      create table t2 ( id int, name string );

      INSERT OVERWRITE TABLE t2 SELECT t.* FROM t1 t;

      select * from t2;

      select count from t2;

      – done

      quit;

      ---------------

      Testing Procedure : Comment/Uncomment the lines for table t2

      1. th.zip
        2 kB
        Naga Vijay

        Activity

        Hide
        Naga Vijay added a comment -

        Modified the title of this JIRA to reflect the issue after further testing.

        Here's the test setup -

        > find . -type f
        ./th12a/testHiveMetastore.hql
        ./th12a/testHiveMetastore.sh
        ./th12a/testHiveMetastore.txt
        ./th12b/testHiveMetastore.hql
        ./th12b/testHiveMetastore.sh

        > Ran script in th12a dir - noticed derby.log and TempStatsStore dir being created in th12a dir
        > Ran script in th12b dir - table t2 has data, so MySQL Metastore is used ; verified in HDFS as well

        So, the issue is - derby.log and TempStatsStore dir are created in th12a dir when using MySQL Metastore

        Show
        Naga Vijay added a comment - Modified the title of this JIRA to reflect the issue after further testing. Here's the test setup - > find . -type f ./th12a/testHiveMetastore.hql ./th12a/testHiveMetastore.sh ./th12a/testHiveMetastore.txt ./th12b/testHiveMetastore.hql ./th12b/testHiveMetastore.sh > Ran script in th12a dir - noticed derby.log and TempStatsStore dir being created in th12a dir > Ran script in th12b dir - table t2 has data, so MySQL Metastore is used ; verified in HDFS as well So, the issue is - derby.log and TempStatsStore dir are created in th12a dir when using MySQL Metastore
        Hide
        Naga Vijay added a comment -

        Steps to test ...

        0. have a proper hadoop, hive (with mysql metastore) setup
        1. unzip attached th.zip
        2. run bash script in th12a dir - notice derby.log and TempStatsStore dir creation in th12a dir - this is the issue
        3. run bash script in th12b dir - to verify table t2 has data in it (you can verify in HDFS as well)

        Issue is - derby.log and TempStatsStore dir are created when using mysql metastore.

        Show
        Naga Vijay added a comment - Steps to test ... 0. have a proper hadoop, hive (with mysql metastore) setup 1. unzip attached th.zip 2. run bash script in th12a dir - notice derby.log and TempStatsStore dir creation in th12a dir - this is the issue 3. run bash script in th12b dir - to verify table t2 has data in it (you can verify in HDFS as well) Issue is - derby.log and TempStatsStore dir are created when using mysql metastore.
        Hide
        Naga Vijay added a comment -

        Not a Problem, as we can avoid creation of derby.log and TempStatsStore by setting hive.stats.autogather to false.

        Show
        Naga Vijay added a comment - Not a Problem, as we can avoid creation of derby.log and TempStatsStore by setting hive.stats.autogather to false.

          People

          • Assignee:
            Unassigned
            Reporter:
            Naga Vijay
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development