Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-1894

HDFS Data replication cannot be initiated independent of Oozie server location

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: trunk, 0.10
    • Component/s: general
    • Labels:
      None

      Description

      The HDFS mirroring scripts allow replication between two clusters.
      Currently, even though the UI allows the replication in any direction between clusters independent of which cluster the Falcon and Oozie servers belong to this is not observed and the source cluster is always the cluster with Oozie/Falcon server.

      Steps to reproduce:
      1) Define both clusters in Falcon server on cluster 2
      2) Set up HDFS mirroring in Falcon server on cluster 2 from Cluster 1 to Cluster 2 and set to run on Oozie server of Cluster 2

      Result:
      Falcon will replicate data from Cluster 2 to Cluster 2

      Cause:
      In hdfs-replication-workflow.xml, the source dir should be defined like the target dir by including clusterfs:
      https://github.com/apache/falcon/blob/master/addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml#L63

      <arg>${drSourceDir}</arg>
      should be
      <arg>${drSourceClusterFS}${drSourceDir}</arg>

      like

      https://github.com/apache/falcon/blob/master/addons/recipes/hdfs-replication/src/main/resources/hdfs-replication-workflow.xml#L65

        Attachments

          Activity

            People

            • Assignee:
              sowmyaramesh Sowmya Ramesh
              Reporter:
              abush Alex Bush
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: