Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1052

HadoopAccessorService.createFileSystem throws exception in map-reduce action, failing workflow.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.3.0, trunk
    • Fix Version/s: 3.3.0
    • Component/s: core
    • Labels:
      None

      Description

      HadoopAccessorService.createFileSystem checks OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED in jobconf, (which is passed as argument), and if false or non-exist, throws exception.
      ------
      if (!conf.getBoolean(OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, false)) {
      throw new HadoopAccessorException(ErrorCode.E0903);
      }
      ------

      this is troublesome when it's called from LauncherMapper.hasIdSwap, since it passes new conf as argument, which doesn't include OOZIE_HADOOP_ACCESSOR_SERVICE_CREATED, so always hitting exception mentioned above
      -------
      FileSystem fs = Services.get().get(HadoopAccessorService.class).createFileSystem(user, p.toUri(),new Configuration());
      --------

      this problem becomes visible to user as workflow failure, when in map-reduce action, oozie fails to get external jobID from counters, then accessing to HDFS to get it.

      the above-mentioned change in HadoopAccessorService.createFileSystem is introduced in 3.2, it is not present in 3.1.4.

      this needs to be fixed. also question is what was original intention of the change.

        Attachments

        1. OOZIE-1052.patch
          1 kB
          Ryota Egashira

          Activity

            People

            • Assignee:
              egashira Ryota Egashira
              Reporter:
              egashira Ryota Egashira
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: