Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3205

Read multiple chunks directly from FSInputChecker subclass into user buffers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.22.0
    • 0.21.0
    • fs
    • None
    • Reviewed

    Description

      Implementations of FSInputChecker and FSOutputSummer like DFS do not have access to full user buffer. At any time DFS can access only up to 512 bytes even though user usually reads with a much larger buffer (often controlled by io.file.buffer.size). This requires implementations to double buffer data if an implementation wants to read or write larger chunks of data from underlying storage.

      We could separate changes for FSInputChecker and FSOutputSummer into two separate jiras.

      Attachments

        1. hadoop-3205.txt
          8 kB
          Todd Lipcon
        2. hadoop-3205.txt
          9 kB
          Todd Lipcon
        3. hadoop-3205.txt
          11 kB
          Todd Lipcon
        4. hadoop-3205.txt
          11 kB
          Todd Lipcon
        5. hadoop-3205.txt
          19 kB
          Todd Lipcon

        Issue Links

          Activity

            People

              tlipcon Todd Lipcon
              rangadi Raghu Angadi
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: