Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2246

CoordinatorInputCheckCommand does not behave properly when har file is one of data dependency and doesn't exist

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:
      None

      Description

      when har file doesn't exist, FileSystem.createFileSystem in HadoopAccessorService throw exception, failing CoordActionInputCheck.
      thus, even thought there are other data dependencies which already exist, it is not reflected on DB.
      coordinator job cannot start until the har file becomes available anyway, and once available, this error doesn't happen, so basic functionality is fine, but it's misleading.

      2014-03-13 22:00:00,051 WARN CallableQueueService$CallableWrapper:542 [pool-2-thread-288] - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] exception callable [coord_action_input], E1021: Coord Action Input Check Error: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      org.apache.oozie.command.CommandException: E1021: Coord Action Input Check Error: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:182)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:67)
      at org.apache.oozie.command.XCommand.call(XCommand.java:280)
      at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.io.IOException: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:493)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkListOfPaths(CoordActionInputCheckXCommand.java:459)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkResolvedUris(CoordActionInputCheckXCommand.java:429)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkInput(CoordActionInputCheckXCommand.java:259)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:132)
      ... 6 more
      Caused by: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:430)
      at org.apache.oozie.dependency.FSURIHandler.getFileSystem(FSURIHandler.java:134)
      at org.apache.oozie.dependency.FSURIHandler.exists(FSURIHandler.java:99)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:488)
      ... 10 more
      Caused by: java.io.IOException: Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har
      at org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:139)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2160)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:303)
      at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:422)
      at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:420)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1284)
      at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
      
      1. OOZIE-2246-v4.patch
        3 kB
        Satish Subhashrao Saley
      2. OOZIE-2246-v3.patch
        3 kB
        Ryota Egashira
      3. OOZIE-2246-v2.patch
        3 kB
        Ryota Egashira
      4. OOZIE-2246.patch
        4 kB
        Ryota Egashira

        Activity

        Hide
        rohini Rohini Palaniswamy added a comment -

        Can we add the check to FSURIHandler exists method instead of coordinputcheck ?

        Show
        rohini Rohini Palaniswamy added a comment - Can we add the check to FSURIHandler exists method instead of coordinputcheck ?
        Hide
        egashira Ryota Egashira added a comment -

        yes, putting into FSURIhandler is cleaner. changed it

        Show
        egashira Ryota Egashira added a comment - yes, putting into FSURIhandler is cleaner. changed it
        Hide
        rohini Rohini Palaniswamy added a comment -

        Can you do the following minor change to avoid any possible NPEs.

        if (ErrorCode.E0902.equals(e.getErrorCode())
        && (e.getMessage() != null && e.getMessage().indexOf("Invalid path for the Har Filesystem. No index file") != -1))

        Show
        rohini Rohini Palaniswamy added a comment - Can you do the following minor change to avoid any possible NPEs. if (ErrorCode.E0902.equals(e.getErrorCode()) && (e.getMessage() != null && e.getMessage().indexOf("Invalid path for the Har Filesystem. No index file") != -1))
        Hide
        rohini Rohini Palaniswamy added a comment -

        +1

        Show
        rohini Rohini Palaniswamy added a comment - +1
        Hide
        hadoopqa Hadoop QA added a comment -

        Testing JIRA OOZIE-2246

        Cleaning local git workspace

        ----------------------------

        +1 PATCH_APPLIES
        +1 CLEAN
        +1 RAW_PATCH_ANALYSIS
        . +1 the patch does not introduce any @author tags
        . +1 the patch does not introduce any tabs
        . +1 the patch does not introduce any trailing spaces
        . +1 the patch does not introduce any line longer than 132
        . +1 the patch does adds/modifies 1 testcase(s)
        +1 RAT
        . +1 the patch does not seem to introduce new RAT warnings
        +1 JAVADOC
        . +1 the patch does not seem to introduce new Javadoc warnings
        +1 COMPILE
        . +1 HEAD compiles
        . +1 patch compiles
        . +1 the patch does not seem to introduce new javac warnings
        +1 BACKWARDS_COMPATIBILITY
        . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations
        . +1 the patch does not modify JPA files
        -1 TESTS
        . Tests run: 1763
        . Tests failed: 5
        . Tests errors: 0

        . The patch failed the following testcases:

        . testForNoDuplicates(org.apache.oozie.event.TestEventGeneration)
        . testKill(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
        . testCatchupJob(org.apache.oozie.command.coord.TestAbandonedCoordChecker)
        . testSamplers(org.apache.oozie.util.TestMetricsInstrumentation)
        . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor)

        +1 DISTRO
        . +1 distro tarball builds with the patch

        ----------------------------
        -1 Overall result, please check the reported -1(s)

        The full output of the test-patch run is available at

        . https://builds.apache.org/job/oozie-trunk-precommit-build/2740/

        Show
        hadoopqa Hadoop QA added a comment - Testing JIRA OOZIE-2246 Cleaning local git workspace ---------------------------- +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS . +1 the patch does not introduce any @author tags . +1 the patch does not introduce any tabs . +1 the patch does not introduce any trailing spaces . +1 the patch does not introduce any line longer than 132 . +1 the patch does adds/modifies 1 testcase(s) +1 RAT . +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC . +1 the patch does not seem to introduce new Javadoc warnings +1 COMPILE . +1 HEAD compiles . +1 patch compiles . +1 the patch does not seem to introduce new javac warnings +1 BACKWARDS_COMPATIBILITY . +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations . +1 the patch does not modify JPA files -1 TESTS . Tests run: 1763 . Tests failed: 5 . Tests errors: 0 . The patch failed the following testcases: . testForNoDuplicates(org.apache.oozie.event.TestEventGeneration) . testKill(org.apache.oozie.command.coord.TestAbandonedCoordChecker) . testCatchupJob(org.apache.oozie.command.coord.TestAbandonedCoordChecker) . testSamplers(org.apache.oozie.util.TestMetricsInstrumentation) . testRecovery(org.apache.oozie.action.hadoop.TestJavaActionExecutor) +1 DISTRO . +1 distro tarball builds with the patch ---------------------------- -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at . https://builds.apache.org/job/oozie-trunk-precommit-build/2740/
        Hide
        puru Purshotam Shah added a comment -

        +1. Test failures are not related.

        Show
        puru Purshotam Shah added a comment - +1. Test failures are not related.
        Hide
        puru Purshotam Shah added a comment -

        committed to trunk.

        Show
        puru Purshotam Shah added a comment - committed to trunk.
        Hide
        satishsaley Satish Subhashrao Saley added a comment -

        Thank you Purshotam Shah

        Show
        satishsaley Satish Subhashrao Saley added a comment - Thank you Purshotam Shah
        Hide
        rkanter Robert Kanter added a comment -

        Closing issue; Oozie 4.3.0 is released.

        Show
        rkanter Robert Kanter added a comment - Closing issue; Oozie 4.3.0 is released.

          People

          • Assignee:
            satishsaley Satish Subhashrao Saley
            Reporter:
            egashira Ryota Egashira
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development