Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-7450

Improve performance for ANALYZE command

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.17.0
    • 1.17.0
    • None

    Description

      In the scope of DRILL-7273 was introduced ANALYZE command for collecting metadata and storing it to the metastore.
      But current implementation uses too much memory and is low-performant. It uses stream aggregate for collecting metadata, but all incoming data should be sorted before producing the aggregation. Memory usage may be reduced by using hash aggregate, so sort may not be produced and it should increase performance.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue DRILL-7433 Allow parallel computation of metadata aggregating

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #1907

          Activity

            People

              volodymyr Vova Vysotskyi
              volodymyr Vova Vysotskyi
              Arina Ielchiieva Arina Ielchiieva
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: