Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7714 Umbrella for usage of native calls to manage OS cache and readahead
  3. HADOOP-7754

Expose file descriptors from Hadoop-wrapped local FileSystems

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 1.2.0, 2.0.2-alpha
    • Component/s: native, performance
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In HADOOP-7714, we determined that using fadvise inside of the MapReduce shuffle can yield very good performance improvements. But many parts of the shuffle are FileSystem-agnostic and thus operate on FSDataInputStreams and RawLocalFileSystems. This JIRA is to figure out how to allow RawLocalFileSystem to expose its FileDescriptor object without unnecessarily polluting the public APIs.

        Attachments

        1. HADOOP-7754_branch-1_rev2.patch
          5 kB
          Ahmed Radwan
        2. HADOOP-7754_branch-1_rev3.patch
          5 kB
          Ahmed Radwan
        3. HADOOP-7754_branch-1_rev4.patch
          6 kB
          Ahmed Radwan
        4. HADOOP-7754_trunk_rev2.patch
          6 kB
          Ahmed Radwan
        5. HADOOP-7754_trunk_rev2.patch
          6 kB
          Ahmed Radwan
        6. HADOOP-7754_trunk_rev3.patch
          6 kB
          Ahmed Radwan
        7. HADOOP-7754_trunk_rev4.patch
          7 kB
          Ahmed Radwan
        8. HADOOP-7754_trunk_rev4.patch
          7 kB
          Ahmed Radwan
        9. HADOOP-7754_trunk_rev4.patch
          7 kB
          Ahmed Radwan
        10. HADOOP-7754_trunk.patch
          5 kB
          Ahmed Radwan
        11. hadoop-7754-0.23.0-hasfd.txt
          4 kB
          Cristina L. Abad
        12. hasfd.txt
          4 kB
          Todd Lipcon
        13. test-patch-hadoop-7754.txt
          168 kB
          Ahmed Radwan

          Issue Links

          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-4511 Add IFile readahead

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                tlipcon Todd Lipcon
                Reporter:
                tlipcon Todd Lipcon
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: