Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-15665

LLAP: OrcFileMetadata objects in cache can impact heap usage

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0
    • llap

    Description

      OrcFileMetadata internally has filestats, stripestats etc which are allocated in heap. On large data sets, this could have an impact on the heap usage and the memory usage by different executors in LLAP.

      Attachments

        1. HIVE-15665.patch
          274 kB
          Sergey Shelukhin
        2. HIVE-15665.01.patch
          331 kB
          Sergey Shelukhin
        3. HIVE-15665.02.patch
          331 kB
          Sergey Shelukhin
        4. HIVE-15665.03.patch
          142 kB
          Sergey Shelukhin
        5. HIVE-15665.04.patch
          142 kB
          Sergey Shelukhin
        6. HIVE-15665.05.patch
          140 kB
          Sergey Shelukhin
        7. HIVE-15665.06.patch
          140 kB
          Sergey Shelukhin
        8. HIVE-15665.07.patch
          140 kB
          Sergey Shelukhin
        9. HIVE-15665.08.patch
          140 kB
          Sergey Shelukhin
        10. HIVE-15665.09.patch
          156 kB
          Sergey Shelukhin
        11. HIVE-15665.10.patch
          158 kB
          Sergey Shelukhin
        12. HIVE-15665.11.patch
          158 kB
          Sergey Shelukhin
        13. HIVE-15665.12.patch
          158 kB
          Sergey Shelukhin

        Issue Links

          blocks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-17524 instrument LLAP metadata cache with separate counters by format

          • Major - Major loss of function.
          • Open
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16233 llap: Query failed with AllocatorOutOfMemoryException

          • Major - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16224 modify off-heap ORC metadata cache after ORC changes

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. ORC-159 expose metadata buffer from metadata reader (for caching)

          • Major - Major loss of function.
          • Open

          Bug - A problem which impairs or prevents the functions of the product. ORC-158 do not assume an array-backed buffer in static ReaderImpl methods

          • Major - Major loss of function.
          • In Progress

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16318 LLAP cache: address some issues in 2.2/2.3

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. ORC-160 expose index read planning in RecordReaderUtils

          • Major - Major loss of function.
          • Closed
          links to

          Web Link RB

          Activity

            People

              sershe Sergey Shelukhin
              rajesh.balamohan Rajesh Balamohan
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: