Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-21948

Implement parallel processing in Pre Upgrade Tool

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.1.0
    • Fix Version/s: 4.0.0
    • Component/s: Transactions
    • Labels:
      None

      Description

      Pre Upgrade Tool scans for all databases and tables in the warehouse sequentially which can be very slow in case of lots of tables.

      Example: It took the process 8-10 hours to complete on ~500k tables.

        Attachments

        1. HIVE-21948.1.patch
          59 kB
          Krisztian Kasa
        2. HIVE-21948.2.patch
          47 kB
          Krisztian Kasa
        3. HIVE-21948.3.patch
          47 kB
          Krisztian Kasa
        4. HIVE-21948.4.patch
          45 kB
          Krisztian Kasa
        5. HIVE-21948.5.patch
          45 kB
          Krisztian Kasa

          Issue Links

            Activity

              People

              • Assignee:
                kkasa Krisztian Kasa
                Reporter:
                kkasa Krisztian Kasa
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: