[HDFS-5731] Refactoring to define interfaces between BM and NN and simplify the flow between them

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: namenode
• Assignee: Unassigned
• Reporter: Amir Langer
• Created: 08/Jan/14 14:03
• Updated: 04/Feb/14 19:32

Description

Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between the two components and defining API interfaces between them. This is done to enable future transformation into a clean RPC protocol.
Logic to calls from Datanodes should be in the BM.
NN should interact with BM using few calls and BM should use the return types as much as possible to pass information to the NN.

The emphasis is on restructuring the request execution flows between the NN and BM in a way that will minimize the latency increase when the BM implementation becomes remote. Namely, the API flows are restructured in a way that BM is called at most once per request.
The two components (NN and BM) still exist in the same VM and share the same memory space.
NN and BM share the same lifecycle – it is assumed that they can't crash/restart separately.
There is still a 1:1 relationship between them.
APIs between NN and BM will be improved to not use the same object instances and turned into a real protocol.
This task should maintain backward compatibility