Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4762

RECOVER PARTITIONS should send new partitions in small batches to HMS

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.5.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Catalog
    • Labels:

      Description

      HMS cannot handle more than 32k partitions in one call. When adding large amount of partitions, Impala should send the new partitions in smaller batches. otherwise HMS could go OOM.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zamsden Zach Amsden
                Reporter:
                jyu@cloudera.com Juan Yu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: