[HDFS-4732] TestDFSUpgradeFromImage fails on Windows due to failure to unpack old image tarball that contains hard links

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.1.0-beta
• Component/s: test
• Hadoop Flags: Reviewed
• Assignee: Chris Nauroth
• Reporter: Chris Nauroth
• Created: 23/Apr/13 15:48
• Updated: 12/May/16 18:19
• Resolved: 23/Apr/13 18:48

Description

On non-Windows, FileUtil#unTar is implemented using external Unix shell commands. On Windows, FileUtil#unTar is implemented in Java using commons-compress. TestDFSUpgradeFromImage uses a testing tarball image of an old HDFS layout version, hadoop-22-dfs-dir.tgz. This file contains hard links. It appears that commons-compress cannot handle the hard links correctly. When it unpacks the file, each hard link ends up as a 0-length file. This causes the test to fail during cluster startup, because the 0-length block files are considered corrupt.

Issue Links

• is part of: HDFS-3602 Enhancements to HDFS for Windows Server and Windows Azure development and runtime environments
• is related to: HDFS-9138 TestDatanodeStartupFixesLegacyStorageIDs fails on Windows due to failure to unpack old image tarball that contains hard links