Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1465

very slow inserts regardless of column number

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • Impala 2.0
    • Product Backlog
    • None
    • # cat /etc/redhat-release
      CentOS release 6.3 (Final) and
      ubuntu/precise/amd64 precise-cdh5

    Description

      Shell build version: Impala Shell v2.0.0-cdh5 (ecf30af) built on Sat Oct 11 13:56:06 PDT 2014)
      [localhost:21000] > create table t(id int);
      Query: create table t(id int)

      Fetched 0 row(s) in 0.04s
      [localhost:21000] > insert into table t values(1);
      Query: insert into table t values(1)
      Inserted 1 row(s) in 11.76s

      profile shows =

      • DML Metastore update finished: 11s315ms (10s050ms);

      Attachments

        Activity

          sergey.benner Sergey Benner added a comment -

          thank you!

          sergey.benner Sergey Benner added a comment - thank you!
          henryr Henry Robinson added a comment -

          It looks like your metastore is being extremely slow - can you try the same create from Hive to see if you get different performance?

          henryr Henry Robinson added a comment - It looks like your metastore is being extremely slow - can you try the same create from Hive to see if you get different performance?
          sergey.benner Sergey Benner added a comment -

          I've tried both Sir, with derby and mysql the machine is quad core 15GBs of ram.
          Derby metastore was default (/var/lib/impala/metastore_db).
          one partition box:
          /dev/xvda1 99G 40G 55G 42% /

          And tried it with remote metastore (mysql).
          Fresh install does the same slow inserts regardless of the machine type either.

          Just tried it with hive:
          hive> select count(1) from t;
          OK
          25
          Time taken: 5.898 seconds, Fetched: 1 row(s)
          hive> insert into table t select 5 from t limit 1;
          OK
          Time taken: 7.216 seconds

          sergey.benner Sergey Benner added a comment - I've tried both Sir, with derby and mysql the machine is quad core 15GBs of ram. Derby metastore was default (/var/lib/impala/metastore_db). one partition box: /dev/xvda1 99G 40G 55G 42% / And tried it with remote metastore (mysql). Fresh install does the same slow inserts regardless of the machine type either. Just tried it with hive: hive> select count(1) from t; OK 25 Time taken: 5.898 seconds, Fetched: 1 row(s) hive> insert into table t select 5 from t limit 1; OK Time taken: 7.216 seconds

          Hi Sergey, it seems that you get the same low perf in all systems. Please upgrade Impala and feel free to re-open if the problem remains.

          ippokratis Ippokratis Pandis added a comment - Hi Sergey, it seems that you get the same low perf in all systems. Please upgrade Impala and feel free to re-open if the problem remains.

          People

            Unassigned Unassigned
            sergey.benner Sergey Benner
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: