Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-24721

Align CacheWriter bufferSize with Llap max alloc

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • llap
    • None

    Description

      Before bumping to ORC-1.6, LLAP_ALLOCATOR_MAX_ALLOC value was also used as the ORC CacheWriter buffer size.

      As per ORC-238, the max bufferSize argument can be up to 2^(3*8 - 1) – i.e., less than 8Mb.

      In HIVE-23553 we decouple the two configuration (LLAP max alloc and CacheWriter buffer size) – this ticket is to investigate if there is a need for these two conf to converge.

      More details: https://github.com/apache/hive/pull/1823#pullrequestreview-575698916

      Attachments

        Issue Links

          is caused by

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-23553 Upgrade ORC version to 1.6.7

          • Major - Major loss of function.
          • Closed

          Activity

            People

              Unassigned Unassigned
              pgaref Panagiotis Garefalakis
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: