Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-2152

Ozone client fails with OOM while writing a large (~300MB) key.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Ozone Client
    • Labels:
      None

      Description

      dd if=/dev/zero of=testfile bs=1024 count=307200
      ozone sh key put /vol1/bucket1/key testfile
      
      Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57) at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) at org.apache.hadoop.hdds.scm.storage.BufferPool.allocateBufferIfNeeded(BufferPool.java:66) at org.apache.hadoop.hdds.scm.storage.BlockOutputStream.write(BlockOutputStream.java:234) at org.apache.hadoop.ozone.client.io.BlockOutputStreamEntry.write(BlockOutputStreamEntry.java:129) at org.apache.hadoop.ozone.client.io.KeyOutputStream.handleWrite(KeyOutputStream.java:211) at org.apache.hadoop.ozone.client.io.KeyOutputStream.write(KeyOutputStream.java:193) at org.apache.hadoop.ozone.client.io.OzoneOutputStream.write(OzoneOutputStream.java:49) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:96) at org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:117) at org.apache.hadoop.ozone.web.ozShell.keys.PutKeyHandler.call(PutKeyHandler.java:55) at picocli.CommandLine.execute(CommandLine.java:1173) at picocli.CommandLine.access$800(CommandLine.java:141)
      

        Attachments

        1. largekey.png
          69 kB
          YiSheng Lien

          Issue Links

            Activity

              People

              • Assignee:
                shashikant Shashikant Banerjee
                Reporter:
                avijayan Aravindan Vijayan
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: