Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2356

Add a way to enable/disable credentials in a workflow

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 4.3.0
    • Component/s: security
    • Labels:
      None

      Description

      Currently, in a Kerberos cluster, you can use the <credentials> section to tell Oozie to get delegation tokens for HCat/Metastore, HS2, HBase, etc. However, this is defined in the workflow.xml, which means that Oozie will always try to get those tokens, even in an non-secure cluster, where it will likely fail. We should add a mechanism to enable/disable getting credentials so that the same workflow.xml can be used in both a secure and non-secure environment; as it is now, you have to maintain two copies of the workflow.xml.

      We can do this fairly simply by adding a job-level property (e.g. oozie.credentials.skip=true) that would skip getting delegation tokens.

      1. OOZIE-2356.001.patch
        9 kB
        Robert Kanter
      2. OOZIE-2356.002.patch
        17 kB
        Robert Kanter

        Activity

        Hide
        rkanter Robert Kanter added a comment -

        The patch adds the job-level oozie.credentials.skip property, which defaults to false. It also makes some improvements in skipping some stuff if we're not going to load credentials to be more efficient. Unit test and docs too.

        Show
        rkanter Robert Kanter added a comment - The patch adds the job-level oozie.credentials.skip property, which defaults to false. It also makes some improvements in skipping some stuff if we're not going to load credentials to be more efficient. Unit test and docs too.
        Hide
        rohini Rohini Palaniswamy added a comment -

        This is a really good one. Can we add couple of enhancements to it?

        • Have a action level override instead of just workflow level. This will be needed if one is talking to different clusters in different actions of the workflow.
        • Have a Oozie server level property to skip credentials. Soon we would like users to be able to run their production workflows in unit test framework with dummy data. It can be useful at that time.
        Show
        rohini Rohini Palaniswamy added a comment - This is a really good one. Can we add couple of enhancements to it? Have a action level override instead of just workflow level. This will be needed if one is talking to different clusters in different actions of the workflow. Have a Oozie server level property to skip credentials. Soon we would like users to be able to run their production workflows in unit test framework with dummy data. It can be useful at that time.
        Hide
        hadoopqa Hadoop QA added a comment -

        Testing JIRA OOZIE-2356

        Cleaning local git workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        +1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . +1 the patch does not introduce any line longer than 132
        . +1 the patch does adds/modifies 1 testcase(s)
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        -1 TESTS - patch does not compile, cannot run testcases
        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/2537/

        Show
        hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2356 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2537/
        Hide
        rkanter Robert Kanter added a comment -

        That's a good idea. The 002 patch allows specifying it at the oozie-site, job, and action level. I've updated the tests and docs accordingly.

        Show
        rkanter Robert Kanter added a comment - That's a good idea. The 002 patch allows specifying it at the oozie-site, job, and action level. I've updated the tests and docs accordingly.
        Hide
        rohini Rohini Palaniswamy added a comment -

        +1

        Show
        rohini Rohini Palaniswamy added a comment - +1
        Hide
        rkanter Robert Kanter added a comment -

        Thanks for the review Rohini. Committed to trunk!

        Show
        rkanter Robert Kanter added a comment - Thanks for the review Rohini. Committed to trunk!
        Hide
        rkanter Robert Kanter added a comment -

        Closing issue; Oozie 4.3.0 is released.

        Show
        rkanter Robert Kanter added a comment - Closing issue; Oozie 4.3.0 is released.

          People

          • Assignee:
            rkanter Robert Kanter
            Reporter:
            rkanter Robert Kanter
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development