Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3205

Read multiple chunks directly from FSInputChecker subclass into user buffers

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Implementations of FSInputChecker and FSOutputSummer like DFS do not have access to full user buffer. At any time DFS can access only up to 512 bytes even though user usually reads with a much larger buffer (often controlled by io.file.buffer.size). This requires implementations to double buffer data if an implementation wants to read or write larger chunks of data from underlying storage.

      We could separate changes for FSInputChecker and FSOutputSummer into two separate jiras.

        Attachments

        1. hadoop-3205.txt
          19 kB
          Todd Lipcon
        2. hadoop-3205.txt
          11 kB
          Todd Lipcon
        3. hadoop-3205.txt
          11 kB
          Todd Lipcon
        4. hadoop-3205.txt
          9 kB
          Todd Lipcon
        5. hadoop-3205.txt
          8 kB
          Todd Lipcon

          Issue Links

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                rangadi Raghu Angadi
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: