Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9845

Update protobuf to 2.5 from 2.4.x

VotersStop watchingWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.5-alpha
    • 2.1.0-beta
    • performance
    • None
    • Reviewed

    Description

      protobuf 2.5 is a bit faster with a new Parse to avoid a builder step and a few other goodies that we'd like to take advantage of over in hbase especially now we are all pb all the time. Unfortunately the protoc generated files are no longer compatible w/ 2.4.1 generated files. Hadoop uses 2.4.1 pb. This latter fact makes it so we cannot upgrade until hadoop does.

      This issue suggests hadoop2 move to protobuf 2.5.

      I can do the patch no prob. if there is interest.

      (When we upgraded our build broke with complaints like the below:

      java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
      	at com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetDatanodeReportRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:21566)
      	at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
      	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
      	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDatanodeReport(ClientNamenodeProtocolTranslatorPB.java:488)
      	at org.apache.hadoop.hdfs.DFSClient.datanodeReport(DFSClient.java:1887)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1798
      ...
      

      More over in HBASE-8165 if interested.

      Attachments

        1. HADOOP-9845.patch
          0.4 kB
          Alejandro Abdelnur
        2. HADOOP-9845.patch
          10 kB
          Alejandro Abdelnur

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            tucu00 Alejandro Abdelnur
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            16 Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment