Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-7117

Consider reading chunk files using MappedByteBuffer

    XMLWordPrintableJSON

Details

    Description

      Running Impala TPC-DS which stresses Ozone DN read path.

      BufferUtils#assignByteBuffers stands out as one of the offender.

      We can experiment with MappedByteBuffer and see if it makes performance better.

      Attachments

        1. ozone_dn-rhel01.ozone.cisco.local.html
          1.97 MB
          Wei-Chiu Chuang
        2. ozone_dn-rhel03.ozone.cisco.local.html
          1.22 MB
          Wei-Chiu Chuang
        3. Screen Shot 2022-08-04 at 7.12.55 AM.png
          663 kB
          Wei-Chiu Chuang

        Issue Links

          fixes

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-9536 Datanode perf: Copying (heap) buffers is costly

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-3748 Consider reusing bytebuffer in FilePerBlockStrategy::readChunk

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-7188 Read chunk files using netty ChunkedNioFile.

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-9536 Datanode perf: Copying (heap) buffers is costly

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #3674

          Activity

            People

              szetszwo Tsz-wo Sze
              weichiu Wei-Chiu Chuang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: