[HIVE-1852] Reduce unnecessary DFSClient.rename() calls

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Assignee: Ning Zhang
• Reporter: Ning Zhang
• Created: 15/Dec/10 21:32
• Updated: 16/Dec/11 23:59
• Resolved: 22/Dec/10 22:46

Description

In Hive client side (MoveTask etc), DFSCleint.rename() is called for every file inside a directory. It is very expensive for a large directory in a busy DFS namenode. We should replace it with a single rename() call on the whole directory.