[FALCON-2028] HDFS extension: Validate and append/remove the scheme://authority for the paths

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: trunk, 0.11
• Component/s: extensions
• Assignee: Sowmya Ramesh
• Reporter: Sowmya Ramesh
• Created: 14/Jun/16 20:53
• Updated: 23/Oct/17 05:00
• Resolved: 12/Aug/16 22:02

Description

In hdfs-mirroring-workflow.xml target EP is added only for target dir.

<arg>-sourcePaths</arg>
<arg>${sourceDir}</arg>
<arg>-targetPath</arg>
<arg>${targetClusterFS}${targetDir}</arg>

in the req, if sourceDir is not sent as fully qualified paths then extension should generate fully qualified paths before submitting the process. For targetDir if passed it should be removed.

e.g.

sourceDir=/user/ambari-qaqa/dr/test/primaryCluster/input1, /user/ambari-qaqa/dr/test/primaryCluster/input2

After processing:
sourceDir=hdfs://240.0.0.10:8020/user/ambari-qaqa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qaqa/dr/test/primaryCluster/input2

Also, if user has already specified the fully qualified path then it should not be added for sourceDir but it should be removed for targetDir

sourceDir=hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input2

After processing:
sourceDir=hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input1, hdfs://240.0.0.10:8020/user/ambari-qa/dr/test/primaryCluster/input2
================================================================

targetDir=hdfs://240.0.0.11:8020/user/ambari-qa/dr/test/backupCluster/outputs

After processing:
targetDir=/user/ambari-qa/dr/test/backupCluster/outputs

Issue Links

• links to: GitHub Pull Request #248