Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1480

Slow DDL statements for tables with large number of partitions

    Details

      Description

      Impala users sometimes report that DDL statements (e.g. alter table partition set location...) are taking multiple seconds (>5) for partitioned tables with large number of partitions. The same operations are significantly faster in hive (sub-second response time).

      Use case:

      • 2 node cluster
      • Single table (24 columns, 3 partition keys) with 2500 partitions
      • alter table foo partition (foo_i = i) set location 'hdfs://.....' takes approximately 5-6sec (0.2 in HIVE)
      • 1 sec delay in the alter stmt is caused by https://issues.apache.org/jira/browse/HIVE-5524

        Attachments

          Activity

            People

            • Assignee:
              dtsirogiannis Dimitris Tsirogiannis
              Reporter:
              dtsirogiannis Dimitris Tsirogiannis
            • Votes:
              4 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: