Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1480

Slow DDL statements for tables with large number of partitions

    XMLWordPrintableJSON

    Details

      Description

      Impala users sometimes report that DDL statements (e.g. alter table partition set location...) are taking multiple seconds (>5) for partitioned tables with large number of partitions. The same operations are significantly faster in hive (sub-second response time).

      Use case:

      • 2 node cluster
      • Single table (24 columns, 3 partition keys) with 2500 partitions
      • alter table foo partition (foo_i = i) set location 'hdfs://.....' takes approximately 5-6sec (0.2 in HIVE)
      • 1 sec delay in the alter stmt is caused by https://issues.apache.org/jira/browse/HIVE-5524

        Attachments

          Activity

            People

            • Assignee:
              dtsirogiannis Dimitris Tsirogiannis
              Reporter:
              dtsirogiannis Dimitris Tsirogiannis
            • Votes:
              4 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: