Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7854

Slow ALTER TABLE and LOAD DATA statements for tables with large number of partitions

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: Impala 2.12.0
    • Fix Version/s: None
    • Component/s: Catalog
    • Environment:
      14 Nodes
      Table in question has 20 columns, 3 partition columns, and 57,475 partitions
    • Epic Color:
      ghx-label-4

      Description

      ALTER TABLE and LOAD DATA statements take minutes (9 minutes for ALTER TABLE and 6 minutes for LOAD DATA) for tables with a large number of partitions.

      Our workaround was to use Hive to perform the LOAD DATA and then perform a REFRESH PARTITION using Impala.

      • 14 Nodes
      • Table in question has 20 columns, 3 partition columns, and 57,475 partitions

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vietn vietn
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: