Hadoop Common
  1. Hadoop Common
  2. HADOOP-3205

Read multiple chunks directly from FSInputChecker subclass into user buffers

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Implementations of FSInputChecker and FSOutputSummer like DFS do not have access to full user buffer. At any time DFS can access only up to 512 bytes even though user usually reads with a much larger buffer (often controlled by io.file.buffer.size). This requires implementations to double buffer data if an implementation wants to read or write larger chunks of data from underlying storage.

      We could separate changes for FSInputChecker and FSOutputSummer into two separate jiras.

      1. hadoop-3205.txt
        19 kB
        Todd Lipcon
      2. hadoop-3205.txt
        11 kB
        Todd Lipcon
      3. hadoop-3205.txt
        11 kB
        Todd Lipcon
      4. hadoop-3205.txt
        9 kB
        Todd Lipcon
      5. hadoop-3205.txt
        8 kB
        Todd Lipcon

        Issue Links

          Activity

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development