Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3056

Implement new mechanism to specify ShareLibs for workflow actions

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 5.0.0
    • Fix Version/s: 5.0.0
    • Component/s: core
    • Labels:
      None

      Description

      OOZIE-2687 introduces the launcher element for workflows:

      <launcher>
                  <memory>1024</memory>
                  <vcores>1</vcores>
                  <java-opts>-Dsome.property=true -XX:+RandomJVMSwitch</java-opts>
                  <env>key=value</env>
                  <queue>root.oozie</queue>
                  <sharelib>spark,hive</sharelib>
              </launcher>
      

      The purpose of this ticket is to discuss and implement new mechanism for handling ShareLib.

      addActionShareLib in JavaActionExecutor should adjusted. Regarding "precedence order":
      if global and an action level launcher and configuration (e.g. oozie.action.sharelib.for.#ACTIONTYPE#) tries to override the sharelib the following should apply:

      config properties defined in an action's <configuration> have priority over an action's <job-xml>, which has priority over the global section's <configuration> and <job-xml>, which has priority over the action defaults in oozie-site, and so on.

      Here we have multiple choices how to handle sharelib:

      • Alternative 1: override sharelib in a way that is consistent with current way of handling Oozie configuration settings.
      • Alternative 2: make sharelib additive
        • For example, if there is a global launcher with sharelib element in a workflow that includes multiple ShareLibs (e.g. A,B), and oozie.action.sharelib.for.#ACTIONTYPE# is also specified for an action's configuration (e.g. C,D), then we take the union of the specified entities (A,B,C,D would be included).
        • It's inconsistent with everything else

        Attachments

        1. OOZIE-3056.001.patch
          19 kB
          Gézapeti
        2. OOZIE-3056.002.patch
          28 kB
          Gézapeti
        3. OOZIE-3056.003.patch
          31 kB
          Gézapeti
        4. OOZIE-3056.004.patch
          32 kB
          Gézapeti
        5. OOZIE-3056.005.patch
          32 kB
          Gézapeti

          Issue Links

            Activity

              People

              • Assignee:
                gezapeti Gézapeti
                Reporter:
                asasvari Attila Sasvári
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: