Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: tools/distcp
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      DistCpOptions has been changed to be constructed with a Builder pattern. This potentially affects applications that invoke DistCp with the Java API.

      Description

      Currently the DistCpOptions class encapsulates all DistCp options, which may be set from command-line (via the OptionsParser) or may be set manually (eg construct an instance and call setters). As there are multiple option fields and more (e.g. HDFS-9868, HDFS-10314) to add, validating them can be cumbersome. Ideally, the DistCpOptions object should be immutable. The benefits are:

      1. DistCpOptions is simple and easier to use and share, plus it scales well
      2. validation is automatic, e.g. manually constructed DistCpOptions gets validated before usage
      3. validation error message is well-defined which does not depend on the order of setters

      This jira is to track the effort of making the DistCpOptions immutable by using a Builder pattern for creation.

        Attachments

        1. HDFS-10533.000.patch
          137 kB
          Mingliang Liu
        2. HDFS-10533.000.patch
          137 kB
          Mingliang Liu
        3. HDFS-10533.001.patch
          140 kB
          Mingliang Liu
        4. HDFS-10533.002.patch
          140 kB
          Mingliang Liu
        5. HDFS-10533.003.patch
          140 kB
          Mingliang Liu
        6. HDFS-10533.004.patch
          142 kB
          Mingliang Liu
        7. HDFS-10533.005.patch
          143 kB
          Mingliang Liu
        8. HDFS-10533.006.patch
          155 kB
          Mingliang Liu
        9. HDFS-10533.007.patch
          155 kB
          Mingliang Liu
        10. HDFS-10533.008.patch
          155 kB
          Mingliang Liu
        11. HDFS-10533.009.patch
          155 kB
          Mingliang Liu
        12. HDFS-10533.010.patch
          163 kB
          Mingliang Liu
        13. HDFS-10533.011.patch
          164 kB
          Mingliang Liu
        14. HDFS-10533.012.patch
          164 kB
          Mingliang Liu

          Issue Links

            Activity

              People

              • Assignee:
                liuml07 Mingliang Liu
                Reporter:
                liuml07 Mingliang Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: