[#HADOOP-2657] Enhancements to DFSClient to support flushing data at any point in time

Hadoop Common

Enhancements to DFSClient to support flushing data at any point in time

Created: 18/Jan/08 06:10 PM Updated: 08/Jul/09 04:42 PM

Return to search

Component/s:

None

Affects Version/s:

None

Fix Version/s:

0.17.0

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
flush.patch	2008-02-20 11:19 PM	dhruba borthakur	23 kB
flush2.patch	2008-02-26 06:55 PM	dhruba borthakur	21 kB
flush3.patch	2008-02-27 09:49 AM	dhruba borthakur	22 kB
flush4.patch	2008-02-29 06:23 AM	dhruba borthakur	21 kB
flush5.patch	2008-03-07 12:08 AM	dhruba borthakur	23 kB
flush6.patch	2008-03-07 09:42 PM	dhruba borthakur	23 kB
flush7.patch	2008-03-10 05:46 AM	dhruba borthakur	23 kB
flush8.patch	2008-03-10 08:14 AM	dhruba borthakur	24 kB
flush9.patch	2008-03-10 09:27 PM	dhruba borthakur	25 kB

Issue Links:

Reference

This issue relates to:

HDFS-24 FSDataOutputStream should flush last partial CRC chunk

This issue is related to:

~~HADOOP-1700~~ Append to files in HDFS

Release Note:	A new API DFSOututStream.flush() flushes all outstanding data to the pipeline of datanodes.
Resolution Date:	11/Mar/08 09:12 PM

Description

« Hide

The HDFS Append Design (~~HADOOP-1700~~) requires that there be a public API to flush data written to a HDFS file that can be invoked by an application. This API (popularly referred to a fflush(OutputStream)) will ensure that data written to the DFSOutputStream is flushed to datanodes and any required metadata is persisted on Namenode.

This API has to handle the case when the client decides to flush after writing data that is not a exact multiple of io.bytes.per.checksum.

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#636104	Tue Mar 11 21:13:29 UTC 2008	dhruba	~~HADOOP-2657~~. A flush call on the DFSOutputStream flushes the last partial CRC chunk too. (dhruba)
				Files Changed
				MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DataNode.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/io/SequenceFile.java MODIFY /hadoop/core/trunk/src/test/org/apache/hadoop/dfs/TestFileAppend.java MODIFY /hadoop/core/trunk/CHANGES.txt MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/fs/FSOutputSummer.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/dfs/DFSClient.java MODIFY /hadoop/core/trunk/src/java/org/apache/hadoop/fs/FSDataOutputStream.java