Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8443

Document dfs.namenode.service.handler.count in hdfs-site.xml

        Details

        • Type: Improvement
        • Status: Resolved
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 2.8.0, 3.0.0-alpha1
        • Component/s: documentation
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          When dfs.namenode.servicerpc-address is configured, NameNode launches an extra RPC server to handle requests from non-client nodes. dfs.namenode.service.handler.count specifies the number of threads for the server but the parameter is not documented anywhere.

          I found a mail for asking about the parameter. http://mail-archives.apache.org/mod_mbox/hadoop-user/201505.mbox/%3CE0D5A619-BDEA-44D2-81EB-C32B8464133D%40gmail.com%3E

          1. Text File
            HDFS-8443.1.patch
            1.0 kB
            J.Andreina
          2. Text File
            HDFS-8443.2.patch
            1.0 kB
            J.Andreina
          3. Text File
            HDFS-8443.3.patch
            1 kB
            J.Andreina

            Activity

            Ascending order - Click to sort in descending order
            Hide
            Permalink
            andreina J.Andreina added a comment -

            I would like update the patch.

            Show
            andreina J.Andreina added a comment - I would like update the patch.
            Hide
            Permalink
            andreina J.Andreina added a comment -

            Attached an initial patch.
            Please review.

            Show
            andreina J.Andreina added a comment - Attached an initial patch. Please review.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -



            -1 overall



            Vote Subsystem Runtime Comment
            0 pre-patch 14m 44s Pre-patch trunk compilation is healthy.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 javac 7m 34s There were no new javac warning messages.
            +1 javadoc 9m 41s There were no new javadoc warning messages.
            +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
            +1 whitespace 0m 0s The patch has no lines that end in whitespace.
            +1 install 1m 36s mvn install still works.
            +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
            +1 native 3m 16s Pre-build of native portion
            -1 hdfs tests 164m 13s Tests failed in hadoop-hdfs.
                202m 4s  



            Reason Tests
            Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate



            Subsystem Report/Notes
            Patch URL http://issues.apache.org/jira/secure/attachment/12734104/HDFS-8443.1.patch
            Optional Tests javadoc javac unit
            git revision trunk / ce53c8e
            hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11060/artifact/patchprocess/testrun_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11060/testReport/
            Java 1.7.0_55
            uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11060/console

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 44s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 34s There were no new javac warning messages. +1 javadoc 9m 41s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 native 3m 16s Pre-build of native portion -1 hdfs tests 164m 13s Tests failed in hadoop-hdfs.     202m 4s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFileTruncate Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734104/HDFS-8443.1.patch Optional Tests javadoc javac unit git revision trunk / ce53c8e hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11060/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11060/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11060/console This message was automatically generated.
            Hide
            Permalink
            arpitagarwal Arpit Agarwal added a comment -

            Thanks for fixing this J.Andreina. I suggest a grammatical fix. +1 otherwise.

            thread that listens --> threads that listen.

            Show
            arpitagarwal Arpit Agarwal added a comment - Thanks for fixing this J.Andreina . I suggest a grammatical fix. +1 otherwise. thread that listens --> threads that listen .
            Hide
            Permalink
            andreina J.Andreina added a comment -

            Thanks Arpit Agarwal for the review comments.
            Please review the updated patch.

            Show
            andreina J.Andreina added a comment - Thanks Arpit Agarwal for the review comments. Please review the updated patch.
            Hide
            Permalink
            ajisakaa Akira Ajisaka added a comment -

            Thanks J.Andreina for taking this issue. Some comments:

            dfs.namenode.handler.count

            • Would you document that if dfs.namenode.servicerpc-address is not configured, the threads handle requests from all the nodes?

            dfs.namenode.service.handler.count

            • Would you document that the parameter is valid only if dfs.namenode.servicerpc-address is configured?
            • The threads handle requests from BackupNode and SecondaryNode as well, so "DataNodes" should be "DataNodes and all other non-client nodes".
            Show
            ajisakaa Akira Ajisaka added a comment - Thanks J.Andreina for taking this issue. Some comments: dfs.namenode.handler.count Would you document that if dfs.namenode.servicerpc-address is not configured, the threads handle requests from all the nodes? dfs.namenode.service.handler.count Would you document that the parameter is valid only if dfs.namenode.servicerpc-address is configured? The threads handle requests from BackupNode and SecondaryNode as well, so "DataNodes" should be "DataNodes and all other non-client nodes".
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -



            -1 overall



            Vote Subsystem Runtime Comment
            0 pre-patch 14m 36s Pre-patch trunk compilation is healthy.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 javac 7m 32s There were no new javac warning messages.
            +1 javadoc 9m 37s There were no new javadoc warning messages.
            +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
            +1 whitespace 0m 0s The patch has no lines that end in whitespace.
            +1 install 1m 33s mvn install still works.
            +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
            +1 native 3m 17s Pre-build of native portion
            -1 hdfs tests 166m 34s Tests failed in hadoop-hdfs.
                204m 7s  



            Reason Tests
            Failed unit tests hadoop.hdfs.web.TestWebHDFS



            Subsystem Report/Notes
            Patch URL http://issues.apache.org/jira/secure/attachment/12734322/HDFS-8443.2.patch
            Optional Tests javadoc javac unit
            git revision trunk / fb6b38d
            hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11072/artifact/patchprocess/testrun_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11072/testReport/
            Java 1.7.0_55
            uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11072/console

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 36s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 native 3m 17s Pre-build of native portion -1 hdfs tests 166m 34s Tests failed in hadoop-hdfs.     204m 7s   Reason Tests Failed unit tests hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12734322/HDFS-8443.2.patch Optional Tests javadoc javac unit git revision trunk / fb6b38d hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11072/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11072/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11072/console This message was automatically generated.
            Hide
            Permalink
            andreina J.Andreina added a comment -

            Thanks Akira Ajisaka for reviewing.
            Updated the patch as per the comments.
            Please review.

            Show
            andreina J.Andreina added a comment - Thanks Akira Ajisaka for reviewing. Updated the patch as per the comments. Please review.
            Hide
            Permalink
            hadoopqa Hadoop QA added a comment -



            -1 overall



            Vote Subsystem Runtime Comment
            0 pre-patch 14m 49s Pre-patch trunk compilation is healthy.
            +1 @author 0m 0s The patch does not contain any @author tags.
            -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
            +1 javac 7m 32s There were no new javac warning messages.
            +1 javadoc 9m 34s There were no new javadoc warning messages.
            +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
            +1 whitespace 0m 0s The patch has no lines that end in whitespace.
            +1 install 1m 33s mvn install still works.
            +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
            +1 native 3m 14s Pre-build of native portion
            +1 hdfs tests 162m 1s Tests passed in hadoop-hdfs.
                199m 45s  



            Subsystem Report/Notes
            Patch URL http://issues.apache.org/jira/secure/attachment/12735302/HDFS-8443.3.patch
            Optional Tests javadoc javac unit
            git revision trunk / 9a3d617
            hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11128/artifact/patchprocess/testrun_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11128/testReport/
            Java 1.7.0_55
            uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11128/console

            This message was automatically generated.

            Show
            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 49s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 32s There were no new javac warning messages. +1 javadoc 9m 34s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 33s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. +1 native 3m 14s Pre-build of native portion +1 hdfs tests 162m 1s Tests passed in hadoop-hdfs.     199m 45s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12735302/HDFS-8443.3.patch Optional Tests javadoc javac unit git revision trunk / 9a3d617 hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11128/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11128/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11128/console This message was automatically generated.
            Hide
            Permalink
            ajisakaa Akira Ajisaka added a comment -

            +1, committing this shortly.

            Show
            ajisakaa Akira Ajisaka added a comment - +1, committing this shortly.
            Hide
            Permalink
            ajisakaa Akira Ajisaka added a comment -

            Committed this to trunk and branch-2. Thanks J.Andreina for the contribution.

            Show
            ajisakaa Akira Ajisaka added a comment - Committed this to trunk and branch-2. Thanks J.Andreina for the contribution.
            Hide
            Permalink
            hudson Hudson added a comment -

            FAILURE: Integrated in Hadoop-trunk-Commit #7923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7923/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Show
            hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7923 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7923/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #212 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/212/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #942 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/942/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Hide
            Permalink
            hudson Hudson added a comment -

            FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Show
            hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2140 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2140/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Hide
            Permalink
            hudson Hudson added a comment -

            FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/201/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Show
            hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #201 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/201/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            Hide
            Permalink
            hudson Hudson added a comment -

            SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/)
            HDFS-8443. Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e)

            • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
            • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
            Show
            hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk-Java8 #210 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/210/ ) HDFS-8443 . Document dfs.namenode.service.handler.count in hdfs-site.xml. Contributed by J.Andreina. (aajisaka: rev d725dd8af682f0877cf523744d9801174b727f4e) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

              People

              • Assignee:
                andreina J.Andreina
                Reporter:
                ajisakaa Akira Ajisaka
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development