Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2741

dfs.datanode.max.xcievers missing in 0.20.205.0

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Document and raise the maximum allowed transfer threads on a DataNode to 4096. This helps Apache HBase in particular.

      Description

      The dfs.datanode.max.xcievers configuration directive is missing in the hdfs-default.xml and documentation.

      1. HDFS-2741-branch1-1.patch
        0.5 kB
        Markus Jelsma
      2. HDFS-2741-branch1-2.patch
        0.5 kB
        Markus Jelsma

        Issue Links

          Activity

          Show
          Markus Jelsma added a comment - As discussed in the following thread: http://search-hadoop.com/m/I5uSBI9xLY&subj=Re+I+O+errors+reading+task+output+on+20+205+0
          Hide
          Harsh J added a comment -

          This is same as HDFS-1866 but targets branch-1 alone.

          Show
          Harsh J added a comment - This is same as HDFS-1866 but targets branch-1 alone.
          Hide
          Harsh J added a comment -

          Markus,

          Thanks for filing this! Would you like to checkout branch-1 and provide a patch as well?

          Show
          Harsh J added a comment - Markus, Thanks for filing this! Would you like to checkout branch-1 and provide a patch as well?
          Hide
          Markus Jelsma added a comment -

          Patch adding dfs.datanode.max.transfer.threads to hdfs-default.xml.

          Show
          Markus Jelsma added a comment - Patch adding dfs.datanode.max.transfer.threads to hdfs-default.xml.
          Hide
          Harsh J added a comment -

          Markus,

          In branch-1 (formerly branch-0.20), the property is named as dfs.datanode.max.xcievers. Could you amend your patch to that effect?

          Also perhaps you may want to add a note about this property under src/docs/ too somewhere relevant (public pages)?

          Show
          Harsh J added a comment - Markus, In branch-1 (formerly branch-0.20), the property is named as dfs.datanode.max.xcievers. Could you amend your patch to that effect? Also perhaps you may want to add a note about this property under src/docs/ too somewhere relevant (public pages)?
          Hide
          Markus Jelsma added a comment -

          Here, but without a note on the setting in the documentation. I've been looking at 0.22 public docs but nothing really comes to mind to write about this setting.

          Show
          Markus Jelsma added a comment - Here, but without a note on the setting in the documentation. I've been looking at 0.22 public docs but nothing really comes to mind to write about this setting.
          Hide
          Harsh J added a comment -

          Markus,

          I think its mostly only HBase that needs this raised anyway. Thanks for the patch, it looks great, committing in a few

          Show
          Harsh J added a comment - Markus, I think its mostly only HBase that needs this raised anyway. Thanks for the patch, it looks great, committing in a few
          Hide
          Harsh J added a comment -

          Committed to branch-1. Thanks for your time and contributions Markus! Looking forward to more improvements and reports from you

          Show
          Harsh J added a comment - Committed to branch-1. Thanks for your time and contributions Markus! Looking forward to more improvements and reports from you
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop-1.1.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

            People

            • Assignee:
              Unassigned
              Reporter:
              Markus Jelsma
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development