Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6044

Add property for setting the NFS look up time for users

        Details

        • Type: Improvement Improvement
        • Status: Closed
        • Priority: Minor Minor
        • Resolution: Fixed
        • Affects Version/s: None
        • Fix Version/s: 2.4.0
        • Component/s: nfs
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          Currently NFS gateway refresh the user account every 15 minutes. Add a property to make it tunable in different environments.

          1. Text File
            HDFS-6044.001.patch
            2 kB
            Brandon Li
          2. Text File
            HDFS-6044.002.patch
            3 kB
            Brandon Li
          3. Text File
            HDFS-6044.003.patch
            4 kB
            Brandon Li

            Issue Links

            is related to

            Bug - A problem which impairs or prevents the functions of the product. HADOOP-10612 NFS failed to refresh the user group id mapping table

            • Major - Major loss of function.
            • Closed

              Activity

              Ascending order - Click to sort in descending order
              Hide
              Permalink
              Hadoop QA added a comment -

              -1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12632373/HDFS-6044.001.patch
              against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              -1 tests included. The patch doesn't appear to include any new or modified tests.
              Please justify why no new tests are needed for this patch.
              Also please list what manual steps were performed to verify this patch.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 javadoc. There were no new javadoc warning messages.

              +1 eclipse:eclipse. The patch built with eclipse:eclipse.

              -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

              -1 release audit. The applied patch generated 1 release audit warnings.

              +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-nfs.

              +1 contrib tests. The patch passed contrib unit tests.

              Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//testReport/
              Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt
              Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-nfs.html
              Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//console

              This message is automatically generated.

              Show
              Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12632373/HDFS-6044.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. -1 release audit . The applied patch generated 1 release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-nfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//artifact/trunk/patchprocess/patchReleaseAuditProblems.txt Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-nfs.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6284//console This message is automatically generated.
              Hide
              Permalink
              Brandon Li added a comment -

              Uploaded a new patch to fix the findbugs warning and add a unit test.

              Show
              Brandon Li added a comment - Uploaded a new patch to fix the findbugs warning and add a unit test.
              Hide
              Permalink
              Hadoop QA added a comment -

              +1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12632666/HDFS-6044.002.patch
              against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 1 new or modified test files.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 javadoc. There were no new javadoc warning messages.

              +1 eclipse:eclipse. The patch built with eclipse:eclipse.

              +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-nfs.

              +1 contrib tests. The patch passed contrib unit tests.

              Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6301//testReport/
              Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6301//console

              This message is automatically generated.

              Show
              Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12632666/HDFS-6044.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-nfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6301//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6301//console This message is automatically generated.
              Hide
              Permalink
              Jing Zhao added a comment -

              The patch looks good to me. Some minor comments:

              1. We may want to define the constant string somewhere in the following code? 
                "+    timeout = conf.getLong("hadoop.nfs3.userupdate.ms", TIMEOUT_DEFAULT);"
              2. Can we declare timeout as final and initialize it in the two constructors?

              Other than that +1.

              Show
              Jing Zhao added a comment - The patch looks good to me. Some minor comments: We may want to define the constant string somewhere in the following code? "+ timeout = conf.getLong(" hadoop.nfs3.userupdate.ms ", TIMEOUT_DEFAULT);" Can we declare timeout as final and initialize it in the two constructors? Other than that +1.
              Hide
              Permalink
              Brandon Li added a comment -

              Uploaded a new patch to address Jing's comments.

              Later on, the new property will be moved to a different location as discussed in HDFS-6050.

              Show
              Brandon Li added a comment - Uploaded a new patch to address Jing's comments. Later on, the new property will be moved to a different location as discussed in HDFS-6050 .
              Hide
              Permalink
              Hadoop QA added a comment -

              +1 overall. Here are the results of testing the latest attachment
              http://issues.apache.org/jira/secure/attachment/12632738/HDFS-6044.003.patch
              against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              +1 tests included. The patch appears to include 1 new or modified test files.

              +1 javac. The applied patch does not increase the total number of javac compiler warnings.

              +1 javadoc. There were no new javadoc warning messages.

              +1 eclipse:eclipse. The patch built with eclipse:eclipse.

              +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

              +1 release audit. The applied patch does not increase the total number of release audit warnings.

              +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-nfs.

              +1 contrib tests. The patch passed contrib unit tests.

              Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6304//testReport/
              Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6304//console

              This message is automatically generated.

              Show
              Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12632738/HDFS-6044.003.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-nfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6304//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6304//console This message is automatically generated.
              Hide
              Permalink
              Brandon Li added a comment -

              Thank you, Jing. I've committed the patch.

              Show
              Brandon Li added a comment - Thank you, Jing. I've committed the patch.
              Hide
              Permalink
              Hudson added a comment -

              SUCCESS: Integrated in Hadoop-trunk-Commit #5267 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5267/)
              HDFS-6044. Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622)

              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java
              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java
              • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5267 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5267/ ) HDFS-6044 . Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622 ) /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              Hudson added a comment -

              FAILURE: Integrated in Hadoop-Hdfs-trunk #1693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1693/)
              HDFS-6044. Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622)

              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java
              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java
              • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1693/ ) HDFS-6044 . Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622 ) /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              Hudson added a comment -

              SUCCESS: Integrated in Hadoop-Yarn-trunk #501 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/501/)
              HDFS-6044. Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622)

              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java
              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java
              • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #501 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/501/ ) HDFS-6044 . Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622 ) /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Hide
              Permalink
              Hudson added a comment -

              SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1718 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1718/)
              HDFS-6044. Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622)

              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java
              • /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java
              • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
              Show
              Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1718 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1718/ ) HDFS-6044 . Add property for setting the NFS look up time for users. Contributed by Brandon Li (brandonli: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1574622 ) /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/nfs/nfs3/IdUserGroup.java /hadoop/common/trunk/hadoop-common-project/hadoop-nfs/src/test/java/org/apache/hadoop/nfs/nfs3/TestIdUserGroup.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

                People

                • Assignee:
                  Brandon Li
                  Reporter:
                  Brandon Li
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  4 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Development