Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18269

LLAP: Fast llap io with slow processing pipeline can lead to OOM

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • None
    • None

    Description

      pendingData linked list in Llap IO elevator (LlapRecordReader.java) may grow indefinitely when Llap IO is faster than processing pipeline. Since we don't have backpressure to slow down the IO, this can lead to indefinite growth of pending data leading to severe GC pressure and eventually lead to OOM.

      This specific instance of LLAP was running on HDFS on top of EBS volume backed by SSD. The query that triggered this is issue was ANALYZE STATISTICS .. FOR COLUMNS which also gather bitvectors. Fast IO and Slow processing case.

      Attachments

        1. Screen Shot 2017-12-13 at 1.15.16 AM.png
          103 kB
          Prasanth Jayachandran
        2. HIVE-18269.1.patch
          5 kB
          Prasanth Jayachandran
        3. HIVE-18269.bad.patch
          5 kB
          Sergey Shelukhin
        4. HIVE-18269.01.patch
          31 kB
          Sergey Shelukhin
        5. HIVE-18269.02.patch
          31 kB
          Sergey Shelukhin
        6. HIVE-18269.03.patch
          31 kB
          Sergey Shelukhin

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. HIVE-13913 LLAP: introduce backpressure to recordreader

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link RB

          Activity

            People

              sershe Sergey Shelukhin
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: