Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4909

Avoid protocol buffer RPC namespace clashes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
    • Fix Version/s: 2.5.0
    • Component/s: datanode, journal-node, namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The revised protocol buffer support seems to be compiling for me when using the protobuf-c cross-compiler. However, I still cannot construct a library of the results. This may be a Hadoop issue, or could be an issue with the protobuf-c cross-compiler. What I see are a bunch of these when attempting to link the resulting .o files:

      /home/common/hadoop/hadoop-common/foo/obj/DatanodeProtocol.pb-c.o: In function `hadoop_hdfsreport_bad_blocks_request_proto_init':
      DatanodeProtocol.pb-c.c.text+0x2bb4): multiple definition of `hadoop_hdfsreport_bad_blocks_request_proto_init'
      /home/common/hadoop/hadoop-common/foo/obj/ClientNamenodeProtocol.pb-c.o:ClientNamenodeProtocol.pb-c.c.text+0x277d): first defined here

      From what I can see, this is caused by the

      package hadoop.hdfs;

      line in the .proto files, when combined with the later

      import "hdfs.proto";

      This appears to bring a complete copy of the hdfs.proto file into the source code, which then recompiles it - leading to the duplicate symbols.

      I have attached an updated pcreate.pl script that illustrates the problem. Excluding the following .proto files allows all to be successfully built and linked:

      DatanodeProtocol
      ClientNamenodeProtocol
      QJournalProtocol

      HTH
      Ralph

        Attachments

        1. HDFS-4909.001.patch
          2 kB
          Colin McCabe
        2. HDFS-4909.002.patch
          6 kB
          Chris Nauroth
        3. pcreate.pl
          8 kB
          Ralph Castain

          Issue Links

            Activity

              People

              • Assignee:
                cmccabe Colin McCabe
                Reporter:
                rhc Ralph Castain
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: