Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-4454 Ozone Streaming Write Pipeline
  3. HDDS-6144

Optimize parameters improve write performance

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Not A Bug
    • None
    • None
    • None

    Description

      The latest HDDS-4454 has poor write performance. And there will be a lot of writing failures. (I tested 200 concurrent write threads, 128MB per file)

      In HDDS-5851, putBlocks are required every 4 chunks, which may affect performance. Originally we only did putBlock once after we wrote the whole block. So let's change the default here and try to keep putBlocks as low as possible. Tests found a 20% improvement in performance.

      Attachments

        1. screenshot-1.png
          217 kB
          mingchao zhao
        2. screenshot-2.png
          227 kB
          mingchao zhao

        Issue Links

          Activity

            People

              micahzhao mingchao zhao
              micahzhao mingchao zhao
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: