[HADOOP-11762] Enable swift distcp to secure HDFS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0, 2.4.0, 2.5.0, 2.4.1, 2.5.1, 2.6.0
• Fix Version/s: 3.0.0-alpha1
• Component/s: fs/swift
• Assignee: Chen He
• Reporter: Chen He
• Created: 27/Mar/15 01:58
• Updated: 12/May/16 18:22
• Resolved: 21/Jul/15 18:19

Description

Even we can use "dfs -put" or "dfs -cp" to move data between swift and secured HDFS, it will be impractical for moving huge amount of data like 10TB or larger.

Current Hadoop code will result in :"java.lang.IllegalArgumentException: java.net.UnknownHostException: container.swiftdomain"

Since it does not support token feature in SwiftNativeFileSystem right now, it will be reasonable that we override the "getCanonicalServiceName" method like other filesystem extensions (S3FileSystem, S3AFileSystem)

Issue Links

• relates to: HADOOP-11750 distcp fails if we copy data from swift to secure HDFS