Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13039

Add documentation for configuration property ipc.maximum.data.length for controlling maximum RPC message size.

    Details

    • Hadoop Flags:
      Reviewed

      Description

      The RPC server enforces a maximum length on incoming messages. Messages larger than the maximum are rejected immediately. The maximum length can be tuned by setting configuration property ipc.maximum.data.length, but this is not documented in core-site.xml.

      1. HADOOP-13039.000.patch
        1 kB
        Mingliang Liu
      2. HADOOP-13039.001.patch
        0.9 kB
        Mingliang Liu
      3. HADOOP-13039.001.patch
        0.9 kB
        Mingliang Liu
      4. HADOOP-13039.001.patch
        1 kB
        Mingliang Liu

        Issue Links

          Activity

          Hide
          cnauroth Chris Nauroth added a comment -

          This is a follow-up from discussion on HDFS-10312.

          Show
          cnauroth Chris Nauroth added a comment - This is a follow-up from discussion on HDFS-10312 .
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          It'd be good to add a warning that this setting should rarely need to be changed. It merits investigating whether the cause of the long RPC messages can be fixed instead.

          Show
          arpitagarwal Arpit Agarwal added a comment - It'd be good to add a warning that this setting should rarely need to be changed. It merits investigating whether the cause of the long RPC messages can be fixed instead.
          Hide
          liuml07 Mingliang Liu added a comment -

          Initial v0 patch for feedback. Thanks.

          Show
          liuml07 Mingliang Liu added a comment - Initial v0 patch for feedback. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 27s trunk passed
          +1 compile 13m 17s trunk passed with JDK v1.8.0_77
          +1 compile 10m 28s trunk passed with JDK v1.7.0_95
          +1 mvnsite 1m 22s trunk passed
          +1 mvneclipse 0m 20s trunk passed
          +1 javadoc 1m 34s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 34s trunk passed with JDK v1.7.0_95
          +1 mvninstall 1m 4s the patch passed
          +1 compile 9m 31s the patch passed with JDK v1.8.0_77
          +1 javac 9m 31s the patch passed
          +1 compile 7m 24s the patch passed with JDK v1.7.0_95
          +1 javac 7m 24s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 javadoc 1m 8s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 13s the patch passed with JDK v1.7.0_95
          +1 unit 10m 49s hadoop-common in the patch passed with JDK v1.8.0_77.
          +1 unit 10m 20s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          82m 32s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799639/HADOOP-13039.000.patch
          JIRA Issue HADOOP-13039
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux fbddf8371d37 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / af9bdbe
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9121/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9121/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 27s trunk passed +1 compile 13m 17s trunk passed with JDK v1.8.0_77 +1 compile 10m 28s trunk passed with JDK v1.7.0_95 +1 mvnsite 1m 22s trunk passed +1 mvneclipse 0m 20s trunk passed +1 javadoc 1m 34s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 34s trunk passed with JDK v1.7.0_95 +1 mvninstall 1m 4s the patch passed +1 compile 9m 31s the patch passed with JDK v1.8.0_77 +1 javac 9m 31s the patch passed +1 compile 7m 24s the patch passed with JDK v1.7.0_95 +1 javac 7m 24s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 javadoc 1m 8s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 13s the patch passed with JDK v1.7.0_95 +1 unit 10m 49s hadoop-common in the patch passed with JDK v1.8.0_77. +1 unit 10m 20s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 82m 32s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12799639/HADOOP-13039.000.patch JIRA Issue HADOOP-13039 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux fbddf8371d37 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / af9bdbe Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9121/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9121/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Hi Mingliang Liu, I think we can remove this string "as potentially malicious". The patch looks good otherwise. Thanks!

          Show
          arpitagarwal Arpit Agarwal added a comment - Hi Mingliang Liu , I think we can remove this string "as potentially malicious". The patch looks good otherwise. Thanks!
          Hide
          liuml07 Mingliang Liu added a comment -

          Per-offline discussion with Arpit Agarwal, the v1 patch refined the documentation to make it clearer.

          Show
          liuml07 Mingliang Liu added a comment - Per-offline discussion with Arpit Agarwal , the v1 patch refined the documentation to make it clearer.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 pending Jenkins, thanks Mingliang Liu.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 pending Jenkins, thanks Mingliang Liu .
          Hide
          cnauroth Chris Nauroth added a comment -

          +1 from me too. Thank you, Mingliang Liu.

          Show
          cnauroth Chris Nauroth added a comment - +1 from me too. Thank you, Mingliang Liu .
          Hide
          liuml07 Mingliang Liu added a comment -

          Don't know why Jenkins was not triggered. Will upload the same patch again.

          Show
          liuml07 Mingliang Liu added a comment - Don't know why Jenkins was not triggered. Will upload the same patch again.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 0m 3s Docker failed to build yetus/hadoop:fbe3e86.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800588/HADOOP-13039.001.patch
          JIRA Issue HADOOP-13039
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9171/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 0m 3s Docker failed to build yetus/hadoop:fbe3e86. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800588/HADOOP-13039.001.patch JIRA Issue HADOOP-13039 Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9171/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The following NEW packages will be installed:
          bats
          0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
          Need to get 10.5 kB of archives.
          After this operation, 70.7 kB of additional disk space will be used.
          WARNING: The following packages cannot be authenticated!
          bats
          E: There are problems and -y was used without --force-yes
          The command '/bin/sh -c apt-get install -y bats' returned a non-zero code: 100

          Total Elapsed time: 0m 3s

          ERROR: Docker failed to build image.

          Seems the precommit build is failing. Will file a bug for Yetus if it happens again.

          Show
          liuml07 Mingliang Liu added a comment - The following NEW packages will be installed: bats 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. Need to get 10.5 kB of archives. After this operation, 70.7 kB of additional disk space will be used. WARNING: The following packages cannot be authenticated! bats E: There are problems and -y was used without --force-yes The command '/bin/sh -c apt-get install -y bats' returned a non-zero code: 100 Total Elapsed time: 0m 3s ERROR: Docker failed to build image. Seems the precommit build is failing. Will file a bug for Yetus if it happens again.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 15m 39s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 53s trunk passed
          +1 compile 6m 3s trunk passed with JDK v1.8.0_92
          +1 compile 6m 47s trunk passed with JDK v1.7.0_95
          +1 mvnsite 1m 6s trunk passed
          +1 mvneclipse 0m 20s trunk passed
          +1 javadoc 0m 54s trunk passed with JDK v1.8.0_92
          +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 42s the patch passed
          +1 compile 5m 53s the patch passed with JDK v1.8.0_92
          +1 javac 5m 53s the patch passed
          +1 compile 6m 49s the patch passed with JDK v1.7.0_95
          +1 javac 6m 49s the patch passed
          +1 mvnsite 0m 56s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 1s The patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 53s the patch passed with JDK v1.8.0_92
          +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95
          +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.8.0_92.
          +1 unit 7m 35s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          74m 37s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800640/HADOOP-13039.001.patch
          JIRA Issue HADOOP-13039
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml
          uname Linux 23b65dc78f14 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8eadd71
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9177/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9177/console
          Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 39s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 53s trunk passed +1 compile 6m 3s trunk passed with JDK v1.8.0_92 +1 compile 6m 47s trunk passed with JDK v1.7.0_95 +1 mvnsite 1m 6s trunk passed +1 mvneclipse 0m 20s trunk passed +1 javadoc 0m 54s trunk passed with JDK v1.8.0_92 +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 42s the patch passed +1 compile 5m 53s the patch passed with JDK v1.8.0_92 +1 javac 5m 53s the patch passed +1 compile 6m 49s the patch passed with JDK v1.7.0_95 +1 javac 6m 49s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 1s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 53s the patch passed with JDK v1.8.0_92 +1 javadoc 1m 2s the patch passed with JDK v1.7.0_95 +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.8.0_92. +1 unit 7m 35s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 74m 37s Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800640/HADOOP-13039.001.patch JIRA Issue HADOOP-13039 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml uname Linux 23b65dc78f14 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8eadd71 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9177/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9177/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          liuml07 Mingliang Liu added a comment -

          The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

          This is for adding missing documentation for config property. No code change was involved.

          Show
          liuml07 Mingliang Liu added a comment - The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. This is for adding missing documentation for config property. No code change was involved.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 I committed this for 2.7.3. Thanks for the contribution Mingliang Liu and thanks Chris Nauroth for the code review.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 I committed this for 2.7.3. Thanks for the contribution Mingliang Liu and thanks Chris Nauroth for the code review.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9677 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9677/)
          HADOOP-13039. Add documentation for configuration property (arp: rev ea5475d1c125ff96b0650d0182f694380412c0da)

          • hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9677 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9677/ ) HADOOP-13039 . Add documentation for configuration property (arp: rev ea5475d1c125ff96b0650d0182f694380412c0da) hadoop-common-project/hadoop-common/src/main/resources/core-default.xml
          Hide
          liuml07 Mingliang Liu added a comment -

          Thank you Arpit Agarwal and Chris Nauroth for your review and comments.

          Show
          liuml07 Mingliang Liu added a comment - Thank you Arpit Agarwal and Chris Nauroth for your review and comments.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Closing the JIRA as part of 2.7.3 release.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

            People

            • Assignee:
              liuml07 Mingliang Liu
              Reporter:
              cnauroth Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development