Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2578

Oozie example distcp job fails to run within an encrypted zone with checksum match error

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: examples
    • Labels:
      None

      Description

      The oozie example workflow of distcp fails on HDFS encrypted clusters with checksum mismatch error. This is an expected error on KMS clusters so this jira is to make this script work on encrypted clusters as well (we just have to pass additional arguments -update -skipcrccheck to the distcp command).

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -

        Testing JIRA OOZIE-2578

        Cleaning local git workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        -1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . +1 the patch does not introduce any line longer than 132
        . -1 the patch does not add/modify any testcase
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        +1 TESTS
        . Tests run: 1787
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/3000/

        Show
        hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2578 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files +1 TESTS . Tests run: 1787 +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/3000/
        Hide
        rkanter Robert Kanter added a comment -

        Does this do anything we don't want to non-encrypted runs?

        Show
        rkanter Robert Kanter added a comment - Does this do anything we don't want to non-encrypted runs?
        Hide
        pbacsko Peter Bacsko added a comment -

        I tried it on a non-encrypted cluster, no problems.

        Show
        pbacsko Peter Bacsko added a comment - I tried it on a non-encrypted cluster, no problems.
        Hide
        rkanter Robert Kanter added a comment -

        +1

        Show
        rkanter Robert Kanter added a comment - +1
        Hide
        rkanter Robert Kanter added a comment -

        Thanks Peter. Committed to master!

        Show
        rkanter Robert Kanter added a comment - Thanks Peter. Committed to master!
        Hide
        rkanter Robert Kanter added a comment -

        Closing issue; Oozie 4.3.0 is released.

        Show
        rkanter Robert Kanter added a comment - Closing issue; Oozie 4.3.0 is released.

          People

          • Assignee:
            pbacsko Peter Bacsko
            Reporter:
            pbacsko Peter Bacsko
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development