Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7421

Performance Patch: Disable Client-Side Nagle By Default

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      HADOOP-2232 was a valuable contribution, but I would suggest that Hadoop IPC should disable Nagle's algorithm by default, to improve performance system-wide. Nagle just adds delay to network connections, which is unnecessary in the high bandwidth networks that Hadoop uses. As it stands, this becomes an obscure, little known but important performance optimization (especially for HBase), which took us days to track down during a performance tuning exercise. Obviously integrating this optimization into the Hadoop release will require performance testing to validate that it doesn't harm performance, but it should only help.

      I will submit a patch for how to address this in trunk.

        Issue Links

          Activity

          Hide
          sureshms Suresh Srinivas added a comment -

          HADOOP-8069 has some work and more discussion than this jira.

          Show
          sureshms Suresh Srinivas added a comment - HADOOP-8069 has some work and more discussion than this jira.
          Hide
          daryn Daryn Sharp added a comment -

          Regarding concern to harming performance, I recently noticed that MR/Yarn job submission is hardcoded to disable nagle in the conf.

          Show
          daryn Daryn Sharp added a comment - Regarding concern to harming performance, I recently noticed that MR/Yarn job submission is hardcoded to disable nagle in the conf.

            People

            • Assignee:
              Unassigned
              Reporter:
              rbodkin Ron Bodkin
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development