Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2741

dfs.datanode.max.xcievers missing in 0.20.205.0

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Document and raise the maximum allowed transfer threads on a DataNode to 4096. This helps Apache HBase in particular.

      Description

      The dfs.datanode.max.xcievers configuration directive is missing in the hdfs-default.xml and documentation.

      1. HDFS-2741-branch1-1.patch
        0.5 kB
        Markus Jelsma
      2. HDFS-2741-branch1-2.patch
        0.5 kB
        Markus Jelsma

        Issue Links

          Activity

          Markus Jelsma created issue -
          Show
          Markus Jelsma added a comment - As discussed in the following thread: http://search-hadoop.com/m/I5uSBI9xLY&subj=Re+I+O+errors+reading+task+output+on+20+205+0
          Harsh J made changes -
          Field Original Value New Value
          Link This issue is superceded by HDFS-1866 [ HDFS-1866 ]
          Hide
          Harsh J added a comment -

          This is same as HDFS-1866 but targets branch-1 alone.

          Show
          Harsh J added a comment - This is same as HDFS-1866 but targets branch-1 alone.
          Harsh J made changes -
          Target Version/s 1.0.0 [ 12318243 ]
          Hide
          Harsh J added a comment -

          Markus,

          Thanks for filing this! Would you like to checkout branch-1 and provide a patch as well?

          Show
          Harsh J added a comment - Markus, Thanks for filing this! Would you like to checkout branch-1 and provide a patch as well?
          Hide
          Markus Jelsma added a comment -

          Patch adding dfs.datanode.max.transfer.threads to hdfs-default.xml.

          Show
          Markus Jelsma added a comment - Patch adding dfs.datanode.max.transfer.threads to hdfs-default.xml.
          Markus Jelsma made changes -
          Attachment HDFS-2741-branch1-1.patch [ 12509296 ]
          Hide
          Harsh J added a comment -

          Markus,

          In branch-1 (formerly branch-0.20), the property is named as dfs.datanode.max.xcievers. Could you amend your patch to that effect?

          Also perhaps you may want to add a note about this property under src/docs/ too somewhere relevant (public pages)?

          Show
          Harsh J added a comment - Markus, In branch-1 (formerly branch-0.20), the property is named as dfs.datanode.max.xcievers. Could you amend your patch to that effect? Also perhaps you may want to add a note about this property under src/docs/ too somewhere relevant (public pages)?
          Hide
          Markus Jelsma added a comment -

          Here, but without a note on the setting in the documentation. I've been looking at 0.22 public docs but nothing really comes to mind to write about this setting.

          Show
          Markus Jelsma added a comment - Here, but without a note on the setting in the documentation. I've been looking at 0.22 public docs but nothing really comes to mind to write about this setting.
          Markus Jelsma made changes -
          Attachment HDFS-2741-branch1-2.patch [ 12509308 ]
          Hide
          Harsh J added a comment -

          Markus,

          I think its mostly only HBase that needs this raised anyway. Thanks for the patch, it looks great, committing in a few

          Show
          Harsh J added a comment - Markus, I think its mostly only HBase that needs this raised anyway. Thanks for the patch, it looks great, committing in a few
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags Reviewed [ 10343 ]
          Hide
          Harsh J added a comment -

          Committed to branch-1. Thanks for your time and contributions Markus! Looking forward to more improvements and reports from you

          Show
          Harsh J added a comment - Committed to branch-1. Thanks for your time and contributions Markus! Looking forward to more improvements and reports from you
          Harsh J made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 1.1.0 [ 12317959 ]
          Resolution Fixed [ 1 ]
          Harsh J made changes -
          Release Note Document and raise the maximum allowed transfer threads on a DataNode to 4096. This helps Apache HBase in particular.
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop-1.1.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          22h 4m 1 Harsh J 03/Jan/12 16:33
          Patch Available Patch Available Resolved Resolved
          6m 11s 1 Harsh J 03/Jan/12 16:39
          Resolved Resolved Closed Closed
          288d 1h 48m 1 Matt Foley 17/Oct/12 19:27

            People

            • Assignee:
              Unassigned
              Reporter:
              Markus Jelsma
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development