[HDFS-7170] DFSOutputStream should query policies from NameNode

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: HDFS-6581
• Component/s: hdfs-client
• Assignee: Arpit Agarwal
• Reporter: Arpit Agarwal
• Created: 30/Sep/14 05:38
• Updated: 04/Oct/14 01:54

Description

DFSOutputStream creates the default policy suite locally to determine whether the LAZY_PERSIST policy is set on a file so it can pass the appropriate flag to DataNode via writeBlock. It should query the policies from the NameNode. Per feedback from jingzhao on HDFS-7159.

Additionally DFSOutputStream gets the effective policy on the file, it should get the local storage policy since effective policy is subject to change. We may need to extend HdfsFileStatus to expose it.

DFSClient will probably need to cache policies since we can't take the overhead of an additional RPC call to NameNode per create/append.