Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2346

Add Oozie job console url and sub-workflow information like the super-parent id and workflow depth into the 'oozie.job.info' property

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None

      Description

      OOZIE-1658 adds all oozie job information, comma separated, into the job conf when enabled but it is missing information of sub-workflows.

      This jira will add information of sub-workflows like the super-parent-id and sub-workflow depth along with other workflow information into the 'oozie.job.info' in job conf.

      1. OOZIE-2346.005.patch
        13 kB
        Akshay Rai
      2. OOZIE-2346.004.patch
        8 kB
        Akshay Rai
      3. OOZIE-2346.003.patch
        7 kB
        Akshay Rai
      4. OOZIE-2346.002.patch
        6 kB
        Akshay Rai
      5. OOZIE-2346.001.patch
        6 kB
        Akshay Rai

        Issue Links

          Activity

          Hide
          rkanter Robert Kanter added a comment -

          Closing issue; Oozie 4.3.0 is released.

          Show
          rkanter Robert Kanter added a comment - Closing issue; Oozie 4.3.0 is released.
          Hide
          akshayrai09 Akshay Rai added a comment -

          Thanks Purshotam Shah for the review and commit.

          Show
          akshayrai09 Akshay Rai added a comment - Thanks Purshotam Shah for the review and commit.
          Hide
          puru Purshotam Shah added a comment -

          Thanks Akshay for patch. Committed to trunk.

          Show
          puru Purshotam Shah added a comment - Thanks Akshay for patch. Committed to trunk.
          Hide
          akshayrai09 Akshay Rai added a comment -

          Ran tests locally. Here are the results.

          -1 TESTS

          • Tests run: 1694
          • Tests failed: 6
          • Tests errors: 0
          • The patch failed the following testcases:
            • testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
            • testHostname(org.apache.oozie.servlet.TestHostnameFilter)
            • testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
            • testMaterizationLookup(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand)
            • testStreaming(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor)
            • testStreamingConfOverride(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor)
          Show
          akshayrai09 Akshay Rai added a comment - Ran tests locally. Here are the results. -1 TESTS Tests run: 1694 Tests failed: 6 Tests errors: 0 The patch failed the following testcases: testSamplers(org.apache.oozie.util.TestMetricsInstrumentation) testHostname(org.apache.oozie.servlet.TestHostnameFilter) testForNoDuplicates(org.apache.oozie.event.TestEventGeneration) testMaterizationLookup(org.apache.oozie.command.coord.TestCoordMaterializeTransitionXCommand) testStreaming(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor) testStreamingConfOverride(org.apache.oozie.action.hadoop.TestMapReduceActionExecutor)
          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2346

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          +1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . +1 the patch does adds/modifies 1 testcase(s)
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS - patch does not compile, cannot run testcases
          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2585/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2346 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2585/
          Hide
          puru Purshotam Shah added a comment -

          +1 Pending jenkins.
          Looks like precommit didn't run, I will trigger it manually.

          Show
          puru Purshotam Shah added a comment - +1 Pending jenkins. Looks like precommit didn't run, I will trigger it manually.
          Hide
          akshayrai09 Akshay Rai added a comment -

          Patch: OOZIE-2346.005.patch

          Overall changes:
          1. Minor changes in docs.
          2. Added workflow super parent id and worklfow depth along with other workflow specific properties into the job conf when oozie.job.info is enabled.
          3. The callback url wasn't injected into the job conf. Fixed it.
          4. Propagate coordinator and bundle information to sub-workflow's job conf.

          NOTE:

          1. Super Parent Id of a worklfow is
            • The coordinator action id if triggered by a coordinator.
            • The oozie job id of the super parent workflow that triggered it.
          2. Workflow depth is the level of the workflow from the super parent workflow. Depth starts from 0.
          Show
          akshayrai09 Akshay Rai added a comment - Patch: OOZIE-2346 .005.patch Overall changes: 1. Minor changes in docs. 2. Added workflow super parent id and worklfow depth along with other workflow specific properties into the job conf when oozie.job.info is enabled. 3. The callback url wasn't injected into the job conf. Fixed it. 4. Propagate coordinator and bundle information to sub-workflow's job conf. NOTE: Super Parent Id of a worklfow is The coordinator action id if triggered by a coordinator. The oozie job id of the super parent workflow that triggered it. Workflow depth is the level of the workflow from the super parent workflow. Depth starts from 0.
          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2346

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          +1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . +1 the patch does adds/modifies 1 testcase(s)
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS - patch does not compile, cannot run testcases
          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2546/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2346 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS - patch does not compile, cannot run testcases +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2546/
          Hide
          akshayrai09 Akshay Rai added a comment -

          Change:

          • Injecting only the base console url rather than the job console url.
          Show
          akshayrai09 Akshay Rai added a comment - Change: Injecting only the base console url rather than the job console url.
          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2346

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          +1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . +1 the patch does adds/modifies 1 testcase(s)
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 1691
          . Tests failed: 1
          . Tests errors: 0

          . The patch failed the following testcases:

          . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2539/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2346 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1691 . Tests failed: 1 . Tests errors: 0 . The patch failed the following testcases: . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2539/
          Hide
          akshayrai09 Akshay Rai added a comment -

          Added oozie job console url to the oozie.job.info list.

          Show
          akshayrai09 Akshay Rai added a comment - Added oozie job console url to the oozie.job.info list.
          Hide
          akshayrai09 Akshay Rai added a comment -

          Purshotam Shah, this jira injects information about the sub-workflows along with the other oozie job information that you added in OOZIE-1658 to the job conf. Could you take a look at the patch and review it?

          Show
          akshayrai09 Akshay Rai added a comment - Purshotam Shah , this jira injects information about the sub-workflows along with the other oozie job information that you added in OOZIE-1658 to the job conf. Could you take a look at the patch and review it?
          Hide
          akshayrai09 Akshay Rai added a comment -

          Requesting a review.

          Show
          akshayrai09 Akshay Rai added a comment - Requesting a review.
          Hide
          akshayrai09 Akshay Rai added a comment -

          The failed test is not related to the patch.

          Show
          akshayrai09 Akshay Rai added a comment - The failed test is not related to the patch.
          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2346

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          +1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . +1 the patch does adds/modifies 1 testcase(s)
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 1690
          . Tests failed: 1
          . Tests errors: 1

          . The patch failed the following testcases:

          . testAdminInstrumentation(org.apache.oozie.client.TestOozieCLI)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2533/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2346 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1690 . Tests failed: 1 . Tests errors: 1 . The patch failed the following testcases: . testAdminInstrumentation(org.apache.oozie.client.TestOozieCLI) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2533/
          Hide
          akshayrai09 Akshay Rai added a comment -

          Patch OOZIE-2346.001.patch

          • Fixed lines longer than 132 characters.
          Show
          akshayrai09 Akshay Rai added a comment - Patch OOZIE-2346 .001.patch Fixed lines longer than 132 characters.
          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2346

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          -1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . -1 the patch contains 2 line(s) longer than 132 characters
          . +1 the patch does adds/modifies 1 testcase(s)
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 1690
          . Tests failed: 1
          . Tests errors: 1589

          . The patch failed the following testcases:

          . testHostname(org.apache.oozie.servlet.TestHostnameFilter)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2527/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2346 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . -1 the patch contains 2 line(s) longer than 132 characters . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1690 . Tests failed: 1 . Tests errors: 1589 . The patch failed the following testcases: . testHostname(org.apache.oozie.servlet.TestHostnameFilter) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2527/

            People

            • Assignee:
              akshayrai09 Akshay Rai
              Reporter:
              akshayrai09 Akshay Rai
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development