Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-8125

Limit number of files generated by unpartitioned insert

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Frontend
    • Labels:

      Description

      One pitfall of multithreaded execution is that, if implemented naively, the number of files generated by an unpartitioned insert will be multiplied by mt_dop.

      We should provide a mechanism to limit the number of files generated, e.g. limit the number of insert fragment instances (note that there a pre-existing problem with unpartitioned inserts generating too many files).

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: