Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-9911

EC bucket Write Performance improve

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • EC
    • None

    Description

      Improve write performance of EC type bucket. 

      Improve the write performance of EC type bucket, about 5 ~ 7 times for small files, about 2 ~ 3 times for large files.

       

      Single Threaded Client

      4K Write QPS: about 70 -> 360

      ozone freon ommg --operation CREATE_KEY -n 100000  -t 1 --volume=s3v --bucket=bucket-object-ec --duration 60 --size=4k

      Before

      After

       

       

      MultiThreaded Client

      4K Write QPS: about 800 -> 5500

      ozone freon ommg --operation CREATE_KEY -n 100000  -t 50 --volume=s3v --bucket=bucket-object-ec --duration 60 --size=4k

      Before

       

      After

       

      S3G

      4K Write QPS: about 800 -> 5000

      warp put --host localhost:9878 --obj.size=4KiB --duration=1m --concurrent=100 --keep-data --noclear --bucket=bucket-object-ec --access-key=ozone --secret-key=xxxxxxx

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            XiChen ChenXi
            XiChen ChenXi

            Dates

              Created:
              Updated:

              Slack

                Issue deployment