Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8443

Document dfs.namenode.service.handler.count in hdfs-site.xml

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When dfs.namenode.servicerpc-address is configured, NameNode launches an extra RPC server to handle requests from non-client nodes. dfs.namenode.service.handler.count specifies the number of threads for the server but the parameter is not documented anywhere.

      I found a mail for asking about the parameter. http://mail-archives.apache.org/mod_mbox/hadoop-user/201505.mbox/%3CE0D5A619-BDEA-44D2-81EB-C32B8464133D%40gmail.com%3E

      1. HDFS-8443.1.patch
        1.0 kB
        J.Andreina
      2. HDFS-8443.2.patch
        1.0 kB
        J.Andreina
      3. HDFS-8443.3.patch
        1 kB
        J.Andreina

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/201/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/201/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7923/)
        HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

        • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7923/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to trunk and branch-2. Thanks J.Andreina for the contribution.

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Thanks J.Andreina for the contribution.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1, committing this shortly.

        Show
        ajisakaa Akira Ajisaka added a comment - +1, committing this shortly.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 49s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 34s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
        +1 native 3m 14s Pre-build of native portion
        +1 hdfs tests 162m 1s Tests passed in hadoop-hdfs.
            199m 45s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12735302/HDFS-8443.3.patch
        Optional Tests javadoc javac unit
        git revision trunk / 9a3d617
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11128/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11128/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11128/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 49s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 native 3m 14s Pre-build of native portion +1 hdfs tests 162m 1s Tests passed in hadoop-hdfs.     199m 45s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12735302/HDFS-8443.3.patch Optional Tests javadoc javac unit git revision trunk / 9a3d617 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11128/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11128/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11128/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Thanks Akira Ajisaka for reviewing.
        Updated the patch as per the comments.
        Please review.

        Show
        andreina J.Andreina added a comment - Thanks Akira Ajisaka for reviewing. Updated the patch as per the comments. Please review.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 32s There were no new javac warning messages.
        +1 javadoc 9m 37s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 33s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 native 3m 17s Pre-build of native portion
        -1 hdfs tests 166m 34s Tests failed in hadoop-hdfs.
            204m 7s  



        Reason Tests
        Failed unit tests hadoop.hdfs.web.TestWebHDFS



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12734322/HDFS-8443.2.patch
        Optional Tests javadoc javac unit
        git revision trunk / fb6b38d
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11072/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11072/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11072/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 native 3m 17s Pre-build of native portion -1 hdfs tests 166m 34s Tests failed in hadoop-hdfs.     204m 7s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734322/HDFS-8443.2.patch Optional Tests javadoc javac unit git revision trunk / fb6b38d hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11072/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11072/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11072/console This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks J.Andreina for taking this issue. Some comments:

        dfs.namenode.handler.count

        • Would you document that if dfs.namenode.servicerpc-address is not configured, the threads handle requests from all the nodes?

        dfs.namenode.service.handler.count

        • Would you document that the parameter is valid only if dfs.namenode.servicerpc-address is configured?
        • The threads handle requests from BackupNode and SecondaryNode as well, so "DataNodes" should be "DataNodes and all other non-client nodes".
        Show
        ajisakaa Akira Ajisaka added a comment - Thanks J.Andreina for taking this issue. Some comments: dfs.namenode.handler.count Would you document that if dfs.namenode.servicerpc-address is not configured, the threads handle requests from all the nodes? dfs.namenode.service.handler.count Would you document that the parameter is valid only if dfs.namenode.servicerpc-address is configured? The threads handle requests from BackupNode and SecondaryNode as well, so "DataNodes" should be "DataNodes and all other non-client nodes".
        Hide
        andreina J.Andreina added a comment -

        Thanks Arpit Agarwal for the review comments.
        Please review the updated patch.

        Show
        andreina J.Andreina added a comment - Thanks Arpit Agarwal for the review comments. Please review the updated patch.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for fixing this J.Andreina. I suggest a grammatical fix. +1 otherwise.

        thread that listens --> threads that listen.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for fixing this J.Andreina . I suggest a grammatical fix. +1 otherwise. thread that listens --> threads that listen .
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 44s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 34s There were no new javac warning messages.
        +1 javadoc 9m 41s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 36s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 native 3m 16s Pre-build of native portion
        -1 hdfs tests 164m 13s Tests failed in hadoop-hdfs.
            202m 4s  



        Reason Tests
        Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12734104/HDFS-8443.1.patch
        Optional Tests javadoc javac unit
        git revision trunk / ce53c8e
        hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11060/artifact/patchprocess/testrun_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11060/testReport/
        Java 1.7.0_55
        uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11060/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 34s There were no new javac warning messages. +1 javadoc 9m 41s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 164m 13s Tests failed in hadoop-hdfs.     202m 4s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734104/HDFS-8443.1.patch Optional Tests javadoc javac unit git revision trunk / ce53c8e hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11060/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11060/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11060/console This message was automatically generated.
        Hide
        andreina J.Andreina added a comment -

        Attached an initial patch.
        Please review.

        Show
        andreina J.Andreina added a comment - Attached an initial patch. Please review.
        Hide
        andreina J.Andreina added a comment -

        I would like update the patch.

        Show
        andreina J.Andreina added a comment - I would like update the patch.

          People

          • Assignee:
            andreina J.Andreina
            Reporter:
            ajisakaa Akira Ajisaka
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development