Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2154

Non-interleaved checksums would optimize block transfers.

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.18.0
    • None
    • None

    Description

      Currently when a block is transfered to a data-node the client interleaves data chunks with the respective checksums.
      This requires creating an extra copy of the original data in a new buffer interleaved with the crcs.
      We can avoid extra copying if the data and the crc are fed to the socket one after another.

      Attachments

        Issue Links

          Activity

            People

              rangadi Raghu Angadi
              shv Konstantin Shvachko
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: