Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24872 PTF vectorization coverage
  3. HIVE-17164

Vectorization: Support PTF (Part 2: Unbounded Support-- Turn ON by default)

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • None
    • Hive

    Description

      Add disk storage backing. Turn hive.vectorized.execution.ptf.enabled on by default.

      Add hive.vectorized.ptf.max.memory.buffering.batch.count to specify the maximum number of vectorized row batch to buffer in memory before spilling to disk.

      Add hive.vectorized.testing.reducer.batch.size parameter to have the Tez Reducer make small batches for making a lot of key group batches that cause memory buffering and disk storage backing.

      Attachments

        1. HIVE-17164.01.patch
          127 kB
          Matt McCline
        2. HIVE-17164.02.patch
          127 kB
          Matt McCline
        3. HIVE-17164.03.patch
          127 kB
          Matt McCline
        4. HIVE-17164.04.patch
          223 kB
          Matt McCline

        Issue Links

          relates to

          Sub-task - The sub-task of the issue HIVE-16369 Vectorization: Support PTF (Part 1: No Custom Window Framing -- Default Only)

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Sub-task - The sub-task of the issue HIVE-24761 Vectorization: Support PTF - bounded start windows

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB#61190

          Activity

            People

              mmccline Matt McCline
              mmccline Matt McCline
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: