Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-2029

HDFS-8828 requires Falcon to update Snapshot based mirroring extension

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 0.10
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      HDFS-8828 has introduced the following backward incompatible behavior in Hadoop 2.8.0 (as yet unreleased version). "-delete" option in snapshot based distcp is invalid. Distcp utilizes the snapshot diff report to build diff copy list, if Falcon provides the "-delete" option an exception will be thrown unless the method DistCpOptions.validate() is called before running distcp execute method.

      Where as in Hadoop 2.7.*, Falcon is required to provide "-delete" option for snapshot based distcp.

      HDFS snapshot based mirroring extension provided as part of Falcon 0.10 release (and trunk) is supposed to work with Hadoop 2.7.1 and above. To make the code work with both 2.7., 2.8. and future versions, Falcon should call the validate method on distcp options. If the method does not exist (in 2.7.1), Falcon should catch the exception and proceed with distCp.

        Attachments

          Issue Links

          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14267 Make DistCpOptions class immutable

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                bvellanki Balu Vellanki
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: