Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11307

The rpc to portmap service for NFS has hardcoded timeout.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      The NFS service makes rpc call to the portmap but the timeout is hardcoded. Tests on slow virtual machines sometimes fail due to timeout. We should make the timeout configurable, with the same default value as current value.

      1. HDFS-11307.003.patch
        10 kB
        Mukul Kumar Singh
      2. HDFS-11307.002.patch
        10 kB
        Mukul Kumar Singh
      3. HDFS-11307.001.patch
        7 kB
        Mukul Kumar Singh

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11121 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11121/)
        HDFS-11307. The rpc to portmap service for NFS has hardcoded timeout. (jitendra: rev d1d0b3e1fd593d590aaf2e3db8f730a296b20aa1)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/mount/RpcProgramMountd.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/TestMountd.java
        • (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RpcProgram.java
        • (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleUdpClient.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11121 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11121/ ) HDFS-11307 . The rpc to portmap service for NFS has hardcoded timeout. (jitendra: rev d1d0b3e1fd593d590aaf2e3db8f730a296b20aa1) (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/mount/RpcProgramMountd.java (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/test/java/org/apache/hadoop/hdfs/nfs/TestMountd.java (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/RpcProgram.java (edit) hadoop-common-project/hadoop-nfs/src/main/java/org/apache/hadoop/oncrpc/SimpleUdpClient.java (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/nfs3/RpcProgramNfs3.java (edit) hadoop-hdfs-project/hadoop-hdfs-nfs/src/main/java/org/apache/hadoop/hdfs/nfs/conf/NfsConfigKeys.java
        Hide
        jnp Jitendra Nath Pandey added a comment -

        I have committed this to trunk and branch-2. Thanks Mukul Kumar Singh!

        Show
        jnp Jitendra Nath Pandey added a comment - I have committed this to trunk and branch-2. Thanks Mukul Kumar Singh !
        Hide
        jnp Jitendra Nath Pandey added a comment -

        +1 for the latest patch. I will commit it shortly.

        Show
        jnp Jitendra Nath Pandey added a comment - +1 for the latest patch. I will commit it shortly.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 2m 0s Maven dependency ordering for branch
        +1 mvninstall 14m 48s trunk passed
        +1 compile 11m 41s trunk passed
        +1 checkstyle 1m 48s trunk passed
        +1 mvnsite 0m 46s trunk passed
        +1 mvneclipse 0m 37s trunk passed
        +1 findbugs 1m 2s trunk passed
        +1 javadoc 0m 39s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 0m 33s the patch passed
        +1 compile 11m 9s the patch passed
        +1 javac 11m 9s the patch passed
        -0 checkstyle 1m 45s root: The patch generated 1 new + 58 unchanged - 1 fixed = 59 total (was 59)
        +1 mvnsite 0m 46s the patch passed
        +1 mvneclipse 0m 40s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 18s the patch passed
        +1 javadoc 0m 39s the patch passed
        +1 unit 0m 30s hadoop-nfs in the patch passed.
        +1 unit 2m 1s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 37s The patch does not generate ASF License warnings.
        55m 33s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11307
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12847155/HDFS-11307.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 461be6dcfd3d 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a6b06f7
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18160/artifact/patchprocess/diff-checkstyle-root.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18160/testReport/
        modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18160/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 2m 0s Maven dependency ordering for branch +1 mvninstall 14m 48s trunk passed +1 compile 11m 41s trunk passed +1 checkstyle 1m 48s trunk passed +1 mvnsite 0m 46s trunk passed +1 mvneclipse 0m 37s trunk passed +1 findbugs 1m 2s trunk passed +1 javadoc 0m 39s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 0m 33s the patch passed +1 compile 11m 9s the patch passed +1 javac 11m 9s the patch passed -0 checkstyle 1m 45s root: The patch generated 1 new + 58 unchanged - 1 fixed = 59 total (was 59) +1 mvnsite 0m 46s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 18s the patch passed +1 javadoc 0m 39s the patch passed +1 unit 0m 30s hadoop-nfs in the patch passed. +1 unit 2m 1s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 37s The patch does not generate ASF License warnings. 55m 33s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11307 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12847155/HDFS-11307.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 461be6dcfd3d 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a6b06f7 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18160/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18160/testReport/ modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18160/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        msingh Mukul Kumar Singh added a comment -

        Annotated the getter for portmapUdpTimeoutMillis

        Show
        msingh Mukul Kumar Singh added a comment - Annotated the getter for portmapUdpTimeoutMillis
        Hide
        jnp Jitendra Nath Pandey added a comment -

        Since getPortmapUdpTimeoutMillis is used only for testing, lets annotate it with @VisibleForTesting.

        I am +1 otherwise, I will commit once this is addressed followed by a jenkins run.

        Show
        jnp Jitendra Nath Pandey added a comment - Since getPortmapUdpTimeoutMillis is used only for testing, lets annotate it with @VisibleForTesting. I am +1 otherwise, I will commit once this is addressed followed by a jenkins run.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 1m 57s Maven dependency ordering for branch
        +1 mvninstall 16m 53s trunk passed
        +1 compile 13m 25s trunk passed
        +1 checkstyle 1m 57s trunk passed
        +1 mvnsite 0m 55s trunk passed
        +1 mvneclipse 0m 43s trunk passed
        +1 findbugs 1m 11s trunk passed
        +1 javadoc 0m 44s trunk passed
        0 mvndep 0m 17s Maven dependency ordering for patch
        +1 mvninstall 0m 33s the patch passed
        +1 compile 10m 12s the patch passed
        +1 javac 10m 12s the patch passed
        -0 checkstyle 1m 38s root: The patch generated 1 new + 59 unchanged - 1 fixed = 60 total (was 60)
        +1 mvnsite 0m 45s the patch passed
        +1 mvneclipse 0m 37s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 23s the patch passed
        +1 javadoc 0m 38s the patch passed
        +1 unit 0m 32s hadoop-nfs in the patch passed.
        +1 unit 2m 8s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 34s The patch does not generate ASF License warnings.
        59m 10s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11307
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846846/HDFS-11307.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 37a8bf4e8372 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a5ec1e3
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18152/artifact/patchprocess/diff-checkstyle-root.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18152/testReport/
        modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18152/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 57s Maven dependency ordering for branch +1 mvninstall 16m 53s trunk passed +1 compile 13m 25s trunk passed +1 checkstyle 1m 57s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 43s trunk passed +1 findbugs 1m 11s trunk passed +1 javadoc 0m 44s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 0m 33s the patch passed +1 compile 10m 12s the patch passed +1 javac 10m 12s the patch passed -0 checkstyle 1m 38s root: The patch generated 1 new + 59 unchanged - 1 fixed = 60 total (was 60) +1 mvnsite 0m 45s the patch passed +1 mvneclipse 0m 37s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 23s the patch passed +1 javadoc 0m 38s the patch passed +1 unit 0m 32s hadoop-nfs in the patch passed. +1 unit 2m 8s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 59m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11307 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846846/HDFS-11307.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 37a8bf4e8372 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a5ec1e3 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18152/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18152/testReport/ modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18152/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        msingh Mukul Kumar Singh added a comment -

        handled checkstyle issue and added a unit test

        Show
        msingh Mukul Kumar Singh added a comment - handled checkstyle issue and added a unit test
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 1m 58s Maven dependency ordering for branch
        +1 mvninstall 14m 58s trunk passed
        +1 compile 11m 50s trunk passed
        +1 checkstyle 1m 47s trunk passed
        +1 mvnsite 0m 49s trunk passed
        +1 mvneclipse 0m 38s trunk passed
        +1 findbugs 1m 6s trunk passed
        +1 javadoc 0m 39s trunk passed
        0 mvndep 0m 17s Maven dependency ordering for patch
        +1 mvninstall 0m 30s the patch passed
        +1 compile 11m 8s the patch passed
        +1 javac 11m 8s the patch passed
        -0 checkstyle 1m 44s root: The patch generated 10 new + 57 unchanged - 1 fixed = 67 total (was 58)
        +1 mvnsite 0m 47s the patch passed
        +1 mvneclipse 0m 38s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 25s the patch passed
        +1 javadoc 0m 40s the patch passed
        +1 unit 0m 31s hadoop-nfs in the patch passed.
        +1 unit 2m 4s hadoop-hdfs-nfs in the patch passed.
        +1 asflicense 0m 40s The patch does not generate ASF License warnings.
        56m 14s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11307
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846528/HDFS-11307.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d88776f6e803 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9594c35
        Default Java 1.8.0_111
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18125/artifact/patchprocess/diff-checkstyle-root.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18125/testReport/
        modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18125/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 1m 58s Maven dependency ordering for branch +1 mvninstall 14m 58s trunk passed +1 compile 11m 50s trunk passed +1 checkstyle 1m 47s trunk passed +1 mvnsite 0m 49s trunk passed +1 mvneclipse 0m 38s trunk passed +1 findbugs 1m 6s trunk passed +1 javadoc 0m 39s trunk passed 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 0m 30s the patch passed +1 compile 11m 8s the patch passed +1 javac 11m 8s the patch passed -0 checkstyle 1m 44s root: The patch generated 10 new + 57 unchanged - 1 fixed = 67 total (was 58) +1 mvnsite 0m 47s the patch passed +1 mvneclipse 0m 38s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 25s the patch passed +1 javadoc 0m 40s the patch passed +1 unit 0m 31s hadoop-nfs in the patch passed. +1 unit 2m 4s hadoop-hdfs-nfs in the patch passed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 56m 14s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11307 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846528/HDFS-11307.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d88776f6e803 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9594c35 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18125/artifact/patchprocess/diff-checkstyle-root.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18125/testReport/ modules C: hadoop-common-project/hadoop-nfs hadoop-hdfs-project/hadoop-hdfs-nfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18125/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        msingh Mukul Kumar Singh added a comment -

        This patch adds a new configuration key for portmap connection timeout

        Show
        msingh Mukul Kumar Singh added a comment - This patch adds a new configuration key for portmap connection timeout

          People

          • Assignee:
            msingh Mukul Kumar Singh
            Reporter:
            jnp Jitendra Nath Pandey
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development