Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-50

Falcon replication having issue to s3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.4
    • Component/s: general
    • Labels:

      Description

      Overview :- Replication of data from one cluster to s3 is not working.

      Detail:-
      Replication feed was submitted and scheduled through falcon. While only the retention bundle and co-ordinator (in oozie) are kicked up, could not see any sign of replication bundle or co-ordinator in oozie logs. Dont see any error in logs.

      Below is the feed definition :-

      <?xml version="1.0" encoding="UTF-8" standalone="yes"?>
      <feed name="process-name" description="process-name" xmlns="uri:falcon:feed:0.1">
          <frequency>hours(1)</frequency>
          <late-arrival cut-off="hours(6)" />
          <clusters>
              <cluster name="cluster1" type="source">
                  <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
                  <retention limit="days(24)" action="delete"/>
              </cluster>
              <cluster name="cluster2" type="target">
                  <validity start="2013-07-17T00:00Z" end="2034-12-20T08:00Z"/>
                  <retention limit="days(90)" action="delete"/>
                  <locations>
                      <location type="data" path="s3://<bucket-name>/<path-folder>/${YEAR}-${MONTH}-${DAY}-${HOUR}/"/>
                  </locations>
              </cluster>
          </clusters>
          <locations>
              <location type="data" path="/path/to/copy/${YEAR}/${MONTH}/${DAY}/${HOUR}/"/>
          </locations>
          <ACL owner="user" group="group" permission="0x755"/>
          <schema location="/schema/s1" provider="protobuf"/>
          <properties>
              <property name="queueName" value="queueName"/>
              <property name="jobPriority" value="NORMAL"/>
          </properties>
      </feed>
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              biswajit.nayak Biswajit Nayak
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: