Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2246

CoordinatorInputCheckCommand does not behave properly when har file is one of data dependency and doesn't exist

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.3.0
    • None
    • None

    Description

      when har file doesn't exist, FileSystem.createFileSystem in HadoopAccessorService throw exception, failing CoordActionInputCheck.
      thus, even thought there are other data dependencies which already exist, it is not reflected on DB.
      coordinator job cannot start until the har file becomes available anyway, and once available, this error doesn't happen, so basic functionality is fine, but it's misleading.

      2014-03-13 22:00:00,051 WARN CallableQueueService$CallableWrapper:542 [pool-2-thread-288] - USER[-] GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] exception callable [coord_action_input], E1021: Coord Action Input Check Error: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      org.apache.oozie.command.CommandException: E1021: Coord Action Input Check Error: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:182)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:67)
      at org.apache.oozie.command.XCommand.call(XCommand.java:280)
      at org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:175)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.io.IOException: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:493)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkListOfPaths(CoordActionInputCheckXCommand.java:459)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkResolvedUris(CoordActionInputCheckXCommand.java:429)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.checkInput(CoordActionInputCheckXCommand.java:259)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.execute(CoordActionInputCheckXCommand.java:132)
      ... 6 more
      Caused by: org.apache.oozie.service.HadoopAccessorException: E0902: Exception occured: [Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har]
      at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:430)
      at org.apache.oozie.dependency.FSURIHandler.getFileSystem(FSURIHandler.java:134)
      at org.apache.oozie.dependency.FSURIHandler.exists(FSURIHandler.java:99)
      at org.apache.oozie.command.coord.CoordActionInputCheckXCommand.pathExists(CoordActionInputCheckXCommand.java:488)
      ... 10 more
      Caused by: java.io.IOException: Invalid path for the Har Filesystem. No index file in har://aaaa:8020/data/2014031322/archive.har
      at org.apache.hadoop.fs.HarFileSystem.initialize(HarFileSystem.java:139)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2160)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:303)
      at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:422)
      at org.apache.oozie.service.HadoopAccessorService$2.run(HadoopAccessorService.java:420)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1284)
      at org.apache.oozie.service.HadoopAccessorService.createFileSystem(HadoopAccessorService.java:420)
      

      Attachments

        1. OOZIE-2246.patch
          4 kB
          Ryota Egashira
        2. OOZIE-2246-v2.patch
          3 kB
          Ryota Egashira
        3. OOZIE-2246-v3.patch
          3 kB
          Ryota Egashira
        4. OOZIE-2246-v4.patch
          3 kB
          Satish Saley

        Activity

          People

            satishsaley Satish Saley
            egashira Ryota Egashira
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: