[HDFS-4909] Avoid protocol buffer RPC namespace clashes

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta, 3.0.0-alpha1
• Fix Version/s: 2.5.0
• Component/s: datanode, journal-node, namenode
• Target Version/s: 2.5.0
• Hadoop Flags: Reviewed
• Assignee: Colin McCabe
• Reporter: Ralph Castain
• Created: 16/Jun/13 21:16
• Updated: 12/May/16 18:14
• Resolved: 16/Apr/14 23:02

Description

The revised protocol buffer support seems to be compiling for me when using the protobuf-c cross-compiler. However, I still cannot construct a library of the results. This may be a Hadoop issue, or could be an issue with the protobuf-c cross-compiler. What I see are a bunch of these when attempting to link the resulting .o files:

/home/common/hadoop/hadoop-common/foo/obj/DatanodeProtocol.pb-c.o: In function `hadoop_hdfsreport_bad_blocks_request_proto_init':
DatanodeProtocol.pb-c.c.text+0x2bb4): multiple definition of `hadoop_hdfsreport_bad_blocks_request_proto_init'
/home/common/hadoop/hadoop-common/foo/obj/ClientNamenodeProtocol.pb-c.o:ClientNamenodeProtocol.pb-c.c.text+0x277d): first defined here

From what I can see, this is caused by the

package hadoop.hdfs;

line in the .proto files, when combined with the later

import "hdfs.proto";

This appears to bring a complete copy of the hdfs.proto file into the source code, which then recompiles it - leading to the duplicate symbols.

I have attached an updated pcreate.pl script that illustrates the problem. Excluding the following .proto files allows all to be successfully built and linked:

DatanodeProtocol
ClientNamenodeProtocol
QJournalProtocol

HTH
Ralph

Issue Links

• is duplicated by: HDFS-6161 fix HDFS protobuf namespace conflicts
• relates to: HDFS-4866 Protocol buffer support cannot compile under C