Hadoop Common
  1. Hadoop Common
  2. HADOOP-8558

Hadoop RPC does not allow protocol extension with common interfaces.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not a Problem
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Hadoop RPC fails if MyProtocol extends an interface, which is not a VersionedProtocol even if MyProtocol extends also VersionedProtocol. The reason is that Invocation uses Method.getDeclaringClass(), which returns the interface class rather than the class of MyProtocol.
      This is incompatible with former versions.

      1. TestProtocolExtension.java
        3 kB
        Konstantin Shvachko
      2. TestProtocolExtension.java
        3 kB
        Konstantin Shvachko

        Issue Links

          Activity

          Hide
          Konstantin Shvachko added a comment -

          This was introduced in 0.23 by HADOOP-7227.

          Show
          Konstantin Shvachko added a comment - This was introduced in 0.23 by HADOOP-7227 .
          Hide
          Konstantin Shvachko added a comment -

          Here is an example of what I mean.
          TestProtocol extends VersionedProtocol and ProtocolExtension. The latter is not versioned, and rpc fails with

          Tests in error:
          testCalls(org.apache.hadoop.ipc.TestProtocolExtension): java.lang.NoSuchFieldException: versionID
          

          I think there is no reason for this behavior, because TestProtocol is perfectly versioned.
          The solution is to overload in TestProtocol the method declared in ProtocolExtension. (Uncomment the lines in the test to make it pass.) This is inconvenient if ProtocolExtension has a lot of methods.

          This basically prevents from using other existing interfaces to define new protocols.
          And this is incompatible with previous versions of hadoop, as the same test runs fine with e.g. 0.22.

          Solution is simple enough - need get rid of getDeclaringClass() by passing the actual protocol into Invocation constructor. I would like to here if this is done intentionally and has some meaning I don't understand.

          Show
          Konstantin Shvachko added a comment - Here is an example of what I mean. TestProtocol extends VersionedProtocol and ProtocolExtension . The latter is not versioned, and rpc fails with Tests in error: testCalls(org.apache.hadoop.ipc.TestProtocolExtension): java.lang.NoSuchFieldException: versionID I think there is no reason for this behavior, because TestProtocol is perfectly versioned. The solution is to overload in TestProtocol the method declared in ProtocolExtension . (Uncomment the lines in the test to make it pass.) This is inconvenient if ProtocolExtension has a lot of methods. This basically prevents from using other existing interfaces to define new protocols. And this is incompatible with previous versions of hadoop, as the same test runs fine with e.g. 0.22. Solution is simple enough - need get rid of getDeclaringClass() by passing the actual protocol into Invocation constructor. I would like to here if this is done intentionally and has some meaning I don't understand.
          Hide
          Konstantin Shvachko added a comment -

          Turns out that the version can be specified in annotation now. In my example above if I add ProtocolInfo annotation specifying the version like this

          @ProtocolInfo(
              protocolName = "org.apache.hadoop.ipc.TestProtocolExtension$TestProtocol", 
              protocolVersion = TestProtocol.versionID)
          public interface ProtocolExtention {
            void logClassName();
          }
          

          the test will pass. ProtocolInfo was introduced by HADOOP-7524.

          Show
          Konstantin Shvachko added a comment - Turns out that the version can be specified in annotation now. In my example above if I add ProtocolInfo annotation specifying the version like this @ProtocolInfo( protocolName = "org.apache.hadoop.ipc.TestProtocolExtension$TestProtocol" , protocolVersion = TestProtocol.versionID) public interface ProtocolExtention { void logClassName(); } the test will pass. ProtocolInfo was introduced by HADOOP-7524 .
          Hide
          Konstantin Shvachko added a comment -

          Attaching version of the test that passes under current trunk.

          Show
          Konstantin Shvachko added a comment - Attaching version of the test that passes under current trunk.
          Hide
          Konstantin Shvachko added a comment -

          Resolving as not a problem.

          Show
          Konstantin Shvachko added a comment - Resolving as not a problem.

            People

            • Assignee:
              Konstantin Shvachko
              Reporter:
              Konstantin Shvachko
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development