Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2475

Oozie does not cleanup action dir of killed actions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None

      Description

      There is corner case where leaking happens.
      when workflow KillCommand is issued, WfEndXCommand is invoked in the end finally block, and WfEndXCommand.deleteWFDir() kills action Dir (e.g., /user/satish/oozie_satish/123450-15000000000000-oozie_satish-W)
      but, when this happens right before launcher mapper uploading actionData to HDFS, previously deleted actionDir is created again. the actionDir will not be cleaned up after. This leak can fill up namespace quota of user over a period of time.

      The solution is to delete clean up action dir in ActionKillXCommand.

      1. OOZIE-2475-1.patch
        5 kB
        Satish Subhashrao Saley

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -

          Testing JIRA OOZIE-2475

          Cleaning local git workspace

          ----------------------------

          +1 PATCH_APPLIES
          +1 CLEAN
          -1 RAW_PATCH_ANALYSIS
          . +1 the patch does not introduce any @author tags
          . +1 the patch does not introduce any tabs
          . +1 the patch does not introduce any trailing spaces
          . +1 the patch does not introduce any line longer than 132
          . -1 the patch does not add/modify any testcase
          +1 RAT
          . +1 the patch does not seem to introduce new RAT warnings
          +1 JAVADOC
          . +1 the patch does not seem to introduce new Javadoc warnings
          +1 COMPILE
          . +1 HEAD compiles
          . +1 patch compiles
          . +1 the patch does not seem to introduce new javac warnings
          +1 BACKWARDS_COMPATIBILITY
          . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
          . +1 the patch does not modify JPA files
          -1 TESTS
          . Tests run: 1762
          . Tests failed: 3
          . Tests errors: 0

          . The patch failed the following testcases:

          . testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
          . testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
          . testbulkWfKillSuccess(org.apache.oozie.command.wf.TestBulkWorkflowXCommand)

          +1 DISTRO
          . +1 distro tarball builds with the patch

          ----------------------------
          -1 Overall result, please check the reported -1(s)

          The full output of the test-patch run is available at

          . https://builds.apache.org/job/oozie-trunk-precommit-build/2768/

          Show
          hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2475 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . -1 the patch does not add/modify any testcase +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1762 . Tests failed: 3 . Tests errors: 0 . The patch failed the following testcases: . testForNoDuplicates(org.apache.oozie.event.TestEventGeneration) . testSamplers(org.apache.oozie.util.TestMetricsInstrumentation) . testbulkWfKillSuccess(org.apache.oozie.command.wf.TestBulkWorkflowXCommand) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2768/
          Hide
          satishsaley Satish Subhashrao Saley added a comment -

          Test failures are not related

          Show
          satishsaley Satish Subhashrao Saley added a comment - Test failures are not related
          Hide
          rohini Rohini Palaniswamy added a comment -

          +1. Committed to trunk. Thanks for the fix Satish.

          Show
          rohini Rohini Palaniswamy added a comment - +1. Committed to trunk. Thanks for the fix Satish.
          Hide
          rkanter Robert Kanter added a comment -

          Closing issue; Oozie 4.3.0 is released.

          Show
          rkanter Robert Kanter added a comment - Closing issue; Oozie 4.3.0 is released.

            People

            • Assignee:
              satishsaley Satish Subhashrao Saley
              Reporter:
              satishsaley Satish Subhashrao Saley
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development