Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-9903

EC data write as data streams

    XMLWordPrintableJSON

Details

    Description

      Today, EC write is done in a discrete fashion via GRPC where each chunk is modeled as a WriteChunk request. Because of that, we have to implement a big chunk size to archive stream-like performance, and other unnatural improvements like ZeroCopy on GRPC. 

      GRPC and discrete data transfer protocol is not the perfect fit for data transfer. We should move to use native stream, leverage Netty, to implement EC writes. 

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HDDS-4454 Ozone Streaming Write Pipeline

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HDDS-9904 Ozone reads as data streams

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              duongnguyen Duong
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: