Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1480

Slow DDL statements for tables with large number of partitions

    XMLWordPrintableJSON

Details

    Description

      Impala users sometimes report that DDL statements (e.g. alter table partition set location...) are taking multiple seconds (>5) for partitioned tables with large number of partitions. The same operations are significantly faster in hive (sub-second response time).

      Use case:

      • 2 node cluster
      • Single table (24 columns, 3 partition keys) with 2500 partitions
      • alter table foo partition (foo_i = i) set location 'hdfs://.....' takes approximately 5-6sec (0.2 in HIVE)
      • 1 sec delay in the alter stmt is caused by https://issues.apache.org/jira/browse/HIVE-5524

      Attachments

        Activity

          People

            dtsirogiannis Dimitris Tsirogiannis
            dtsirogiannis Dimitris Tsirogiannis
            Votes:
            4 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: