Details

    • Type: Improvement
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.5-alpha
    • Fix Version/s: 2.1.0-beta
    • Component/s: performance
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      protobuf 2.5 is a bit faster with a new Parse to avoid a builder step and a few other goodies that we'd like to take advantage of over in hbase especially now we are all pb all the time. Unfortunately the protoc generated files are no longer compatible w/ 2.4.1 generated files. Hadoop uses 2.4.1 pb. This latter fact makes it so we cannot upgrade until hadoop does.

      This issue suggests hadoop2 move to protobuf 2.5.

      I can do the patch no prob. if there is interest.

      (When we upgraded our build broke with complaints like the below:

      java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
      	at com.google.protobuf.GeneratedMessage.getUnknownFields(GeneratedMessage.java:180)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$GetDatanodeReportRequestProto.getSerializedSize(ClientNamenodeProtocolProtos.java:21566)
      	at com.google.protobuf.AbstractMessageLite.toByteString(AbstractMessageLite.java:49)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.constructRpcRequest(ProtobufRpcEngine.java:149)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:193)
      	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:164)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:83)
      	at com.sun.proxy.$Proxy14.getDatanodeReport(Unknown Source)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getDatanodeReport(ClientNamenodeProtocolTranslatorPB.java:488)
      	at org.apache.hadoop.hdfs.DFSClient.datanodeReport(DFSClient.java:1887)
      	at org.apache.hadoop.hdfs.MiniDFSCluster.waitActive(MiniDFSCluster.java:1798
      ...
      

      More over in HBASE-8165 if interested.

      1. HADOOP-9845.patch
        10 kB
        Alejandro Abdelnur
      2. HADOOP-9845.patch
        0.4 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          stack stack added a comment -

          Assigning Alejandro Abdelnur at his request

          Show
          stack stack added a comment - Assigning Alejandro Abdelnur at his request
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Attached patch that bumps to protobuf JAR 2.5.0.

          In the 2.1.0-beta branch, I've installed the protoc compiler locally, did a full build, started pseudo cluster, run a few jobs, tested httpfs.

          Everything seems fine.

          The patch will fail in trunk because of YARN-885.

          Now, how do we get protoc 2.5.0 in the jenkins boxes?

          Show
          tucu00 Alejandro Abdelnur added a comment - Attached patch that bumps to protobuf JAR 2.5.0. In the 2.1.0-beta branch, I've installed the protoc compiler locally, did a full build, started pseudo cluster, run a few jobs, tested httpfs. Everything seems fine. The patch will fail in trunk because of YARN-885 . Now, how do we get protoc 2.5.0 in the jenkins boxes?
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Marking it as blocker as it will be more difficult for consumer projects, like HBase, which depend on protoc to move to a newer version after beta.

          stack, the failures you seen in Hbase I believe are because your are trying to compile 2.4.0 generated protobuf java wit the 2.5.0 protobuf JAR. The prob should go away if you regenerate your protobuf java code with 2.5.0 (have you considered using the maven-protoc plugin so you don't have to checking generated java code. granted, every hbase developer will have to have the protoc compiler installed).

          Show
          tucu00 Alejandro Abdelnur added a comment - Marking it as blocker as it will be more difficult for consumer projects, like HBase, which depend on protoc to move to a newer version after beta. stack , the failures you seen in Hbase I believe are because your are trying to compile 2.4.0 generated protobuf java wit the 2.5.0 protobuf JAR. The prob should go away if you regenerate your protobuf java code with 2.5.0 (have you considered using the maven-protoc plugin so you don't have to checking generated java code. granted, every hbase developer will have to have the protoc compiler installed).
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596515/HADOOP-9845.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in .

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2941//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2941//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596515/HADOOP-9845.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in . +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2941//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2941//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          test-patch didn't run any testcases because the changes are hadoop-project, otherwise it would have failed due to the mismatch between the protoc version in the jenkinks machines and the protobuf JAR version in the POMs.

          Show
          tucu00 Alejandro Abdelnur added a comment - test-patch didn't run any testcases because the changes are hadoop-project, otherwise it would have failed due to the mismatch between the protoc version in the jenkinks machines and the protobuf JAR version in the POMs.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          updated patch that makes the protoc maven plugin to check/enforce the protoc version in used. the protoc version in used is defined in the hadoop-project POM and it is used for the protobuf JAR dependency version and for the protoc plugin invocation, keeping them in sync.

          I've did positive and negative build tests to ensure the plugin verification is correct.

          Show
          tucu00 Alejandro Abdelnur added a comment - updated patch that makes the protoc maven plugin to check/enforce the protoc version in used. the protoc version in used is defined in the hadoop-project POM and it is used for the protobuf JAR dependency version and for the protoc plugin invocation, keeping them in sync. I've did positive and negative build tests to ensure the plugin verification is correct.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12596622/HADOOP-9845.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2942//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12596622/HADOOP-9845.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2942//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment - - edited

          the build is now failing because protoc 2.4.0 sends the version to sdterr while protoc 2.5.0 sends it to stdout.

          As with this patch we are requiring 2.5.0 I think we are good as I don't expect protoc --version will swing back to stderr in future versions.

          Show
          tucu00 Alejandro Abdelnur added a comment - - edited the build is now failing because protoc 2.4.0 sends the version to sdterr while protoc 2.5.0 sends it to stdout. As with this patch we are requiring 2.5.0 I think we are good as I don't expect protoc --version will swing back to stderr in future versions.
          Hide
          stack stack added a comment -

          Alejandro Abdelnur The errors were encountered at runtime.

          Patch looks good to me. Looks like you fellas generate the protobufs at build time, that you do not check them in. Do you have to ask infra about updating the protoc to 2.5?

          Thanks Alejandro Abdelnur

          Show
          stack stack added a comment - Alejandro Abdelnur The errors were encountered at runtime. Patch looks good to me. Looks like you fellas generate the protobufs at build time, that you do not check them in. Do you have to ask infra about updating the protoc to 2.5? Thanks Alejandro Abdelnur
          Hide
          acmurthy Arun C Murthy added a comment -

          I'm concerned that this is quite late for hadoop-2 and that protobuf-2.5 is fairly new (released May of this year).

          stack Are you guys comfy with 2.5 over at HBase? Good experience so far? Thanks.

          Show
          acmurthy Arun C Murthy added a comment - I'm concerned that this is quite late for hadoop-2 and that protobuf-2.5 is fairly new (released May of this year). stack Are you guys comfy with 2.5 over at HBase? Good experience so far? Thanks.
          Hide
          stack stack added a comment -

          Mighty Arun C Murthy, yes sir, hbase would like this. It would not be the end of the world if it didn't happen but we are now all pb, all the time, so it'd be grand to have the bit of speedup 2.5 promises and the little bits of new utility it brings. If we are to do the upgrade at all, now is the time to do it while both our parent – hadoop – and ourselves are at a major release junction (you w/ your 2.1 and us w/ our 0.96).

          I watched the pb 2.5beta cycle; it went on a good while before they made the release. Here are the current list of issues sorted by date: https://code.google.com/p/protobuf/issues/list?can=2&q=&sort=-modified+-id+status+priority+-type&colspec=ID%20Type%20Status%20Priority%20FixedIn%20Owner%20Summary%20Modified No high priority issues and just a few that are issue w/ 2.5 – don't look related. I'd think that if issue in 2.5 that it would have been found by now given how widely used pb is. Thanks.

          Show
          stack stack added a comment - Mighty Arun C Murthy , yes sir, hbase would like this. It would not be the end of the world if it didn't happen but we are now all pb, all the time, so it'd be grand to have the bit of speedup 2.5 promises and the little bits of new utility it brings. If we are to do the upgrade at all, now is the time to do it while both our parent – hadoop – and ourselves are at a major release junction (you w/ your 2.1 and us w/ our 0.96). I watched the pb 2.5beta cycle; it went on a good while before they made the release. Here are the current list of issues sorted by date: https://code.google.com/p/protobuf/issues/list?can=2&q=&sort=-modified+-id+status+priority+-type&colspec=ID%20Type%20Status%20Priority%20FixedIn%20Owner%20Summary%20Modified No high priority issues and just a few that are issue w/ 2.5 – don't look related. I'd think that if issue in 2.5 that it would have been found by now given how widely used pb is. Thanks.
          Hide
          daryn Daryn Sharp added a comment -

          I'm a bit uneasy, but undecided, about upgrading to a new protobuf at this late hour in the hadoop-2 release. 2.4.1 has 2+ years of proven stability. 2.5's main benefit seems to be ~25% faster parsing performance, so my questions are:

          • Is the parsing improvement for edge cases, or all cases?
          • Is the improvement even measurable at scale? 25% might equate to a micro-optimization.
          • If yes, is it substantial enough to risk switching to a much younger release now, rather than later?
          • Are we sure it works across all development (*nix, osx, windows) environments?
          • Are we sure it plays nice with Eclipse?
          Show
          daryn Daryn Sharp added a comment - I'm a bit uneasy, but undecided, about upgrading to a new protobuf at this late hour in the hadoop-2 release. 2.4.1 has 2+ years of proven stability. 2.5's main benefit seems to be ~25% faster parsing performance, so my questions are: Is the parsing improvement for edge cases, or all cases? Is the improvement even measurable at scale? 25% might equate to a micro-optimization. If yes, is it substantial enough to risk switching to a much younger release now, rather than later? Are we sure it works across all development (*nix, osx, windows) environments? Are we sure it plays nice with Eclipse?
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          i've tested it linux and osx. intellij didnt complain.

          doing the upgrade now during beta seems less risky than doing it later. we can iron out things without making issues in a ga release

          Show
          tucu00 Alejandro Abdelnur added a comment - i've tested it linux and osx. intellij didnt complain. doing the upgrade now during beta seems less risky than doing it later. we can iron out things without making issues in a ga release
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #4249 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4249/)
          HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
          • /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #4249 (See https://builds.apache.org/job/Hadoop-trunk-Commit/4249/ ) HADOOP-9845 . Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Hide
          shv Konstantin Shvachko added a comment -

          I am getting compile error after this patch went in. Both on my local box and on Jenkins. Could you please advise.

          [INFO] --- hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-common ---
          [WARNING] [protoc, --version] failed with error code 1
          [ERROR] protoc, could not get version
          
          Show
          shv Konstantin Shvachko added a comment - I am getting compile error after this patch went in. Both on my local box and on Jenkins. Could you please advise. [INFO] --- hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (compile-protoc) @ hadoop-common --- [WARNING] [protoc, --version] failed with error code 1 [ERROR] protoc, could not get version
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Konstantin Shvachko, there is a thread in all dev aliases, you need o upgrad to protoc 2.5.0

          Show
          tucu00 Alejandro Abdelnur added a comment - Konstantin Shvachko , there is a thread in all dev aliases, you need o upgrad to protoc 2.5.0
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/300/)
          HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
          • /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #300 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/300/ ) HADOOP-9845 . Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1490 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1490/)
          HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
          • /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1490 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1490/ ) HADOOP-9845 . Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #1517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1517/)
          HADOOP-9845. Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281)

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml
          • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml
          • /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1517 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1517/ ) HADOOP-9845 . Update protobuf to 2.5 from 2.4.x. (tucu) (tucu: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1513281 ) /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/pom.xml /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/pom.xml /hadoop/common/trunk/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/protoc/ProtocMojo.java /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Alejandro Abdelnur Will this be propagated to branch-2 and 2.1-beta? Thanks.

          Show
          arpitagarwal Arpit Agarwal added a comment - Alejandro Abdelnur Will this be propagated to branch-2 and 2.1-beta? Thanks.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Committed to (previously) trunk, branch-2, branch-2.1-beta and branch-2.1.0-beta.

          Show
          tucu00 Alejandro Abdelnur added a comment - Committed to (previously) trunk, branch-2, branch-2.1-beta and branch-2.1.0-beta.
          Hide
          mantonov Mikhail Antonov added a comment -

          So I'm guessing, 2.0.5/6 aren't getting this patch?

          Show
          mantonov Mikhail Antonov added a comment - So I'm guessing, 2.0.5/6 aren't getting this patch?
          Hide
          shv Konstantin Shvachko added a comment -

          Mikhail, we can port it to 2.0, but there are things to consider.
          I know a few binary distributions that are based on 2.0.
          And people may consider this as an incompatible change since it can affect particularly their cluster management software.
          Other than that it is just a matter of porting the patch.

          Show
          shv Konstantin Shvachko added a comment - Mikhail, we can port it to 2.0, but there are things to consider. I know a few binary distributions that are based on 2.0. And people may consider this as an incompatible change since it can affect particularly their cluster management software. Other than that it is just a matter of porting the patch.
          Hide
          mantonov Mikhail Antonov added a comment -

          Konstantin, thanks for reply!

          Essentially, I'm building BigTop from sources on Fedora 19 (I know it's not supported officially, but I wanted to give it a try), and found that make tries to build 2.0.6-alpha, which is using protobuf 2.4.*, but it seems fedora 19 only installs protobuf 2.5.0.

          May be it's more appropriate to discuss in Bigtop dev mail list?

          Show
          mantonov Mikhail Antonov added a comment - Konstantin, thanks for reply! Essentially, I'm building BigTop from sources on Fedora 19 (I know it's not supported officially, but I wanted to give it a try), and found that make tries to build 2.0.6-alpha, which is using protobuf 2.4.*, but it seems fedora 19 only installs protobuf 2.5.0. May be it's more appropriate to discuss in Bigtop dev mail list?

            People

            • Assignee:
              tucu00 Alejandro Abdelnur
              Reporter:
              stack stack
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development