The hadoop rpc code especially the code in Server.java is fairly complicated and poorly documented. Everytime I make changes there or try and debug an issue, I have relearn parts of the code. The javadoc needs to be improved.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12608119/hadoop-10044.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3209//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3209//console

This message is automatically generated.

+1 to a javadoc patch, but as someone who has recently done some RPC work, I think we need something that gives an overall view of how to implement RPC services.

Particular troublespots for me

• basics of registering a service, client proxy
• why turning Simple auth on is a good feature during dev
• moving to protobuf as a payload (why, how)
• moving to protbuf RPC with the various relay proxies. Here the why really has to come before the how, as having done this once I think you really need a good reason to invest the effort.
• security, delegation tokens, auth: basic services, then under YARN
• failure modes, testing

YARN is going to enable many more people to write distributed apps, which means more people trying to understand the RPC mechanism

Thanks Sanjay! The javadoc is very helpful to understand the RPC code. Some minor comments about the current patch:

1. In RpcConstants.java,

   /**
     * The Rpc-connection header is as follows 
     * +----------------------------------+
     * |  "hrpc" 4 bytes                  |      
     * +----------------------------------+
     * |  Version (1 byte)                |
     * +----------------------------------+
     * |  Service Class (1 byte)          |
     * +----------------------------------+
     * |  AuthProtocol (1 byte)           |      
     * +----------------------------------+
     */
   

   can be moved to somewhere in Server.java (e.g., as javadoc for Connection or Connection#readAndProcess) ?

2. In Connection#saslProcess, the AccessCotnrolException thrown by
   processSaslMessage will be caught by the outside try-catch, and will be wrapped as a WrappedRpcServerException. Thus we do not need to add it to signature and javadoc.
3. Do we need to javadoc the mechanism about how to create SASL server based on different auth methods?

Besides there're some typos and I just modify them directly on the original patch. Upload a new patch fixing typos.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12608377/HADOOP-10044.20131014.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

+1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3214//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3214//console

This message is automatically generated.

If the new javadocs have preformatted code, it should be wrapped with <pre></pre> tags, so that the generated javadocs look OK too

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12608377/HADOOP-10044.20131014.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

+1 javadoc. The javadoc tool did not generate any warning messages.

+1 eclipse:eclipse. The patch built with eclipse:eclipse.

+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The following test timeouts occurred in hadoop-common-project/hadoop-common:

org.apache.hadoop.http.TestHttpServer

+1 contrib tests. The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3356//testReport/
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3356//console

This message is automatically generated.

The failed test timeout is unrelated (and I also ran it successfully).
Committed.

SUCCESS: Integrated in Hadoop-trunk-Commit #4874 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4874/)
HADOOP-10044 Improve the javadoc of rpc code (sanjay Radia) (sradia: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1550486)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcConstants.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

FAILURE: Integrated in Hadoop-Yarn-trunk #420 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/420/)
HADOOP-10044 Improve the javadoc of rpc code (sanjay Radia) (sradia: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1550486)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcConstants.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

SUCCESS: Integrated in Hadoop-Hdfs-trunk #1611 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1611/)
HADOOP-10044 Improve the javadoc of rpc code (sanjay Radia) (sradia: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1550486)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcConstants.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

FAILURE: Integrated in Hadoop-Mapreduce-trunk #1637 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1637/)
HADOOP-10044 Improve the javadoc of rpc code (sanjay Radia) (sradia: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1550486)

• /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/RpcConstants.java
• /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/ipc/Server.java

Hi Sanjay Radia,

Thanks for your earlier work on this. I found that the commit of this jira is missing from branch-2. May I know if there is any concern of putting this to branch-2?

Thanks.

Closing old tickets that are already shipped in a release.