Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12171

LLAP: BuddyAllocator failures when querying uncompressed data

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • None

    Description

      hive> select sum(l_extendedprice * l_discount) as revenue from testing.lineitem where l_shipdate >= '1993-01-01' and l_shipdate < '1994-01-01' ;
      
      Caused by: org.apache.hadoop.hive.common.io.Allocator$AllocatorOutOfMemoryException: Failed to allocate 4920000; at 0 out of 1
              at org.apache.hadoop.hive.llap.cache.BuddyAllocator.allocateMultiple(BuddyAllocator.java:176)
              at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.preReadUncompressedStream(EncodedReaderImpl.java:882)
              at org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:319)
              at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:413)
              at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:194)
              at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:191)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
              at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:191)
              at org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:74)
              at org.apache.hadoop.hive.common.CallableWithNdc.call(CallableWithNdc.java:37)
              ... 4 more
      

      Attachments

        1. HIVE-12171.other.patch
          13 kB
          Sergey Shelukhin
        2. HIVE-12171.patch
          16 kB
          Sergey Shelukhin
        3. HIVE-12171.01.patch
          22 kB
          Sergey Shelukhin
        4. HIVE-12171.02.patch
          22 kB
          Sergey Shelukhin
        5. HIVE-12171.03.patch
          22 kB
          Sergey Shelukhin

        Issue Links

          Activity

            People

              sershe Sergey Shelukhin
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: