[HADOOP-438] DFS pathname limitation.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1.0, 0.1.1, 0.2.0, 0.2.1, 0.3.0, 0.3.1, 0.3.2, 0.4.0, 0.5.0
• Fix Version/s: 0.7.0
• Assignee: Wendy Chien
• Reporter: Konstantin Shvachko
• Created: 10/Aug/06 01:08
• Updated: 08/Jul/09 16:41
• Resolved: 14/Sep/06 20:13

Description

I was trying to create a deep hierarchy of directories using DFS mkdirs().
When the path to the leaf directory became long (~20000) DFS was still able to create
directories with these names, but UTF8 started truncating long strings resulting in
incorrect logging of namespace edits. That later crashed the namenode during restart,
when it was trying to reproduce file creation logged in the edits file with truncated names.
UTF8 is deprecated now so we will have to replace it with Text.
With UTF8 we should enforce a pathname limit of 0xffff/3 = 21845
With Text it is going to be larger. Not sure what the exact number is.