[#HADOOP-3283] Need a mechanism for data nodes to update generation stamps.

Hadoop Common

Need a mechanism for data nodes to update generation stamps.

Created: 18/Apr/08 11:22 PM Updated: 22/Aug/08 07:50 PM

Return to search

Component/s:

None

Affects Version/s:

None

Fix Version/s:

0.18.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
3283_20080423c.patch	2008-04-24 01:35 AM	Tsz Wo (Nicholas), SZE	23 kB
3283_20080424.patch	2008-04-24 11:30 PM	Tsz Wo (Nicholas), SZE	28 kB
3283_20080425versionID.patch	2008-04-25 09:19 PM	Tsz Wo (Nicholas), SZE	2 kB

Issue Links:

Dependants

This issue blocks:

~~HADOOP-1700~~ Append to files in HDFS

Reference

This issue is related to:

HDFS-282 Serialize ipcPort in DatanodeID instead of DatanodeRegistration and DatanodeInfo

Hadoop Flags:	Reviewed, Incompatible change
Release Note:	« Hide Added an IPC server in DataNode and a new IPC protocol InterDatanodeProtocol. Added conf properties dfs.datanode.ipc.address and dfs.datanode.handler.count with defaults "0.0.0.0:50020" and 3, respectively. Changed the serialization in DatanodeRegistration and DatanodeInfo, and therefore, updated the versionID in ClientProtocol, DatanodeProtocol, NamenodeProtocol. Show » Added an IPC server in DataNode and a new IPC protocol InterDatanodeProtocol. Added conf properties dfs.datanode.ipc.address and dfs.datanode.handler.count with defaults "0.0.0.0:50020" and 3, respectively. Changed the serialization in DatanodeRegistration and DatanodeInfo, and therefore, updated the versionID in ClientProtocol, DatanodeProtocol, NamenodeProtocol.
Resolution Date:	25/Apr/08 09:31 PM

Description

« Hide

For implementing file append feature (~~HADOOP-1700~~), a generation stamp is added to each block. We need a mechanism for data nodes to update generation stamps for lease recovery.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#651699	Fri Apr 25 21:05:01 UTC 2008	dhruba	~~HADOOP-3283~~. The Datanode has a RPC server. It currently supports two RPCs: the first RPC retrives the metadata about a block and the second RPC sets the generation stamp of an existing block. (Tsz Wo (Nicholas), SZE via dhruba)
				Files Changed
				MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DataNode.java MODIFY /hadoop/core/trunk/src/test/org/apache/hadoop/dfs/TestHost2NodesMap.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/FSNamesystem.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeID.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/FSImage.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeRegistration.java ADD /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/BlockMetaDataInfo.java MODIFY /hadoop/core/trunk/src/test/org/apache/hadoop/net/TestNetworkTopology.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DataBlockScanner.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java ADD /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/InterDatanodeProtocol.java MODIFY /hadoop/core/trunk/CHANGES.txt ADD /hadoop/core/trunk/src/test/org/apache/hadoop/dfs/TestInterDatanodeProtocol.java MODIFY /hadoop/core/trunk/src/test/org/apache/hadoop/dfs/TestReplicationPolicy.java MODIFY /hadoop/core/trunk/src/test/org/apache/hadoop/dfs/MiniDFSCluster.java MODIFY /hadoop/core/trunk/conf/hadoop-default.xml

Repository	Revision	Date	User	Message
ASF	#651703	Fri Apr 25 21:29:17 UTC 2008	dhruba	~~HADOOP-3283~~. The Datanode has a RPC server. It currently supports two RPCs: the first RPC retrives the metadata about a block and the second RPC sets the generation stamp of an existing block. (Tsz Wo (Nicholas), SZE via dhruba)
				Files Changed
				MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/NamenodeProtocol.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeProtocol.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/ClientProtocol.java

Repository	Revision	Date	User	Message
ASF	#653264	Sun May 04 19:29:04 UTC 2008	rangadi	~~HADOOP-3337~~. Loading FSEditLog was broken by ~~HADOOP-3283~~ since it changed Writable serialization of DatanodeInfo. This patch handles it. (Tsz Wo (Nicholas), SZE via rangadi)
				Files Changed
				MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeDescriptor.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DatanodeInfo.java MODIFY /hadoop/core/trunk/CHANGES.txt MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/FSEditLog.java