Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-10816

[Hbase-Ozone] Hbase Bulkloading Crashing Datanodes

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Ozone Datanode
    • None

    Description

      Datanode heap was set as 31GB, I've tried 8GB heap also.

      When triggering BulkLoad over and over again on Hbase on top of Ozone, DNs are going down with "Cannot allocate memory" error. No specific error is thrown in the DN logs, its being terminated abruptly.

      Checked the process stderr log, it shows below:

       

      OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f8439ca0000, 453246976, 0) failed; error='Cannot allocate memory' (errno=12)

      cc: ashishk sammichen weichiu 

       

      Attachments

        Issue Links

          is caused by

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-10488 Datanode OOM due to run out of mmap handler

          • Major - Major loss of function.
          • Resolved
          Testing discovered

          Bug - A problem which impairs or prevents the functions of the product. RATIS-2083 Timer thread leak when log level is DEBUG

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              pratyush.bhatt Pratyush Bhatt
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: