Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13465

Design Server.Call to be extensible for unified call queue

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: ipc
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The RPC layer supports QoS but other protocols, ex. webhdfs, are completely unconstrained. Generalizing Server.Call to be extensible with simple changes to the handlers will enable unifying the call queue for multiple protocols.

      1. HADOOP-13465.1.patch
        24 kB
        Daryn Sharp
      2. HADOOP-13465.patch
        24 kB
        Daryn Sharp

        Issue Links

          Activity

          Hide
          daryn Daryn Sharp added a comment -

          Forgot to post patch. No tests due to private internal changes to ipc server.

          Show
          daryn Daryn Sharp added a comment - Forgot to post patch. No tests due to private internal changes to ipc server.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 41s trunk passed
          +1 compile 6m 46s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 19s trunk passed
          +1 javadoc 0m 45s trunk passed
          +1 mvninstall 0m 37s the patch passed
          +1 compile 6m 42s the patch passed
          +1 javac 6m 42s the patch passed
          -0 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 11 new + 185 unchanged - 9 fixed = 196 total (was 194)
          +1 mvnsite 0m 51s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 28s the patch passed
          +1 javadoc 0m 45s the patch passed
          +1 unit 8m 8s hadoop-common in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          38m 16s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824882/HADOOP-13465.patch
          JIRA Issue HADOOP-13465
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 6ec49b117296 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 115ecb5
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 41s trunk passed +1 compile 6m 46s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 19s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 37s the patch passed +1 compile 6m 42s the patch passed +1 javac 6m 42s the patch passed -0 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 11 new + 185 unchanged - 9 fixed = 196 total (was 194) +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 28s the patch passed +1 javadoc 0m 45s the patch passed +1 unit 8m 8s hadoop-common in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 38m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12824882/HADOOP-13465.patch JIRA Issue HADOOP-13465 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 6ec49b117296 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 115ecb5 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10332/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 The change looks straight-forward and has no functional change.
          I will commit it to trunk and check tomorrow's qbt report before committing to branch-2.

          Show
          kihwal Kihwal Lee added a comment - +1 The change looks straight-forward and has no functional change. I will commit it to trunk and check tomorrow's qbt report before committing to branch-2.
          Hide
          kihwal Kihwal Lee added a comment -

          Just committed this to trunk. As I said, I will see the result of qbt tomorrow before committing to branch-2.

          Show
          kihwal Kihwal Lee added a comment - Just committed this to trunk. As I said, I will see the result of qbt tomorrow before committing to branch-2.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10347 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10347/)
          HADOOP-13465. Design Server.Call to be extensible for unified call (kihwal: rev d288a0ba8364d81aacda9f4a21022eecb6dc4e22)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10347 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10347/ ) HADOOP-13465 . Design Server.Call to be extensible for unified call (kihwal: rev d288a0ba8364d81aacda9f4a21022eecb6dc4e22) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Hide
          kihwal Kihwal Lee added a comment -

          Looks like it is breaking TestEditLogJournalFailures in hdfs. I am reverting it from trunk. Daryn Sharp, please investigate.

          Show
          kihwal Kihwal Lee added a comment - Looks like it is breaking TestEditLogJournalFailures in hdfs. I am reverting it from trunk. Daryn Sharp , please investigate.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10349 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10349/)
          Revert "HADOOP-13465. Design Server.Call to be extensible for unified (kihwal: rev 81485dbfc1ffb8daa609be8eb31094cc28646dd3)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10349 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10349/ ) Revert " HADOOP-13465 . Design Server.Call to be extensible for unified (kihwal: rev 81485dbfc1ffb8daa609be8eb31094cc28646dd3) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Hide
          daryn Daryn Sharp added a comment -

          It's not a serious issue. When I added postponed RPC calls, I had to add to accommodate tests that expected ExitUtils exceptions to bubble back to the client. So I added the ability to abort responses and did, um, "clever" things to propagate back a can't-happen-in-reality exception.

          To avoid races, the original call is cloned, updated with the fatal exception, sent. The test failed because I sent the original call's response. This patch changes Connection#doResponse from connection.sendResponse(this) to connection.sendResponse(call).

          Show
          daryn Daryn Sharp added a comment - It's not a serious issue. When I added postponed RPC calls, I had to add to accommodate tests that expected ExitUtils exceptions to bubble back to the client. So I added the ability to abort responses and did, um, "clever" things to propagate back a can't-happen-in-reality exception. To avoid races, the original call is cloned, updated with the fatal exception, sent. The test failed because I sent the original call's response. This patch changes Connection#doResponse from connection.sendResponse(this) to connection.sendResponse(call) .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 37s trunk passed
          +1 compile 6m 52s trunk passed
          +1 checkstyle 0m 25s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 17s trunk passed
          +1 javadoc 0m 44s trunk passed
          +1 mvninstall 0m 38s the patch passed
          +1 compile 6m 43s the patch passed
          +1 javac 6m 43s the patch passed
          -0 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 11 new + 185 unchanged - 9 fixed = 196 total (was 194)
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 12s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 27s the patch passed
          +1 javadoc 0m 44s the patch passed
          +1 unit 8m 8s hadoop-common in the patch passed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          38m 16s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HADOOP-13465
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825532/HADOOP-13465.1.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 2d321e833408 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 81485db
          Default Java 1.8.0_101
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 37s trunk passed +1 compile 6m 52s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 17s trunk passed +1 javadoc 0m 44s trunk passed +1 mvninstall 0m 38s the patch passed +1 compile 6m 43s the patch passed +1 javac 6m 43s the patch passed -0 checkstyle 0m 25s hadoop-common-project/hadoop-common: The patch generated 11 new + 185 unchanged - 9 fixed = 196 total (was 194) +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 27s the patch passed +1 javadoc 0m 44s the patch passed +1 unit 8m 8s hadoop-common in the patch passed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 38m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HADOOP-13465 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12825532/HADOOP-13465.1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2d321e833408 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 81485db Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/artifact/patchprocess/diff-checkstyle-hadoop-common-project_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/10376/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 I've conformed the issue was addressed in the latest version of the patch.

          Show
          kihwal Kihwal Lee added a comment - +1 I've conformed the issue was addressed in the latest version of the patch.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10386 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10386/)
          HADOOP-13465. Design Server.Call to be extensible for unified call (kihwal: rev 76cd81f4b656f0d40a4b2122e15f04ea53d8020b)

          • (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10386 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10386/ ) HADOOP-13465 . Design Server.Call to be extensible for unified call (kihwal: rev 76cd81f4b656f0d40a4b2122e15f04ea53d8020b) (edit) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java
          Hide
          kihwal Kihwal Lee added a comment -

          Committed to trunk through branch-2.8.

          Show
          kihwal Kihwal Lee added a comment - Committed to trunk through branch-2.8.

            People

            • Assignee:
              daryn Daryn Sharp
              Reporter:
              daryn Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development