Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1821

Oozie java action fails due to AlreadyBeingCreatedException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: trunk, 3.3.2
    • Fix Version/s: 4.1.0
    • Component/s: action
    • Labels:
      None
    • Environment:

      Hadoop 0.20.2-cdh3u5
      Oozie server build version: 3.3.2.8

      Description

      2014-04-17 17:13:33,773 WARN ActionStartXCommand:542 - USER[<user_id>] GROUP[-] TOKEN[] APP[<app_name>] JOB[0078484- 140410182733292-oozie-oozi-W] ACTION[0078484-140410182733292-oozie-oozi-W@recordsize] Error starting action [recordsize]. ErrorType [TRANSIENT], ErrorCode [JA009], Message [JA009: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: failed to create file /user/<user-name>/oozie-oozi/0078484-140410182733292-oozie-oozi-W/recordsize--java/action.xml for DFSClient_-1385059579 on client <client ip> because current leaseholder is trying to recreate file.
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.recoverLeaseInternal(FSNamesystem.java:1407)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInternal(FSNamesystem.java:1279)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:1221)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:703)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.create(NameNode.java:682)
      at sun.reflect.GeneratedMethodAccessor27.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1439)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1435)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1433)
      ]

      Earlier the java action had failed due to disk quota exceeded exception which is a transient error. when oozie retried to run the action, later retrials resulted in above exception.

      In org.apache.oozie.action.hadoop,LauncherMapper, if writing action xml fails due to an error, output stream should be closed gracefully.

      OutputStream os = fs.create(new Path(actionDir, ACTION_CONF_XML));
      actionConf.writeXml(os);
      os.close();

        Attachments

        1. OOZIE_1821-V3.patch
          2 kB
          Abhishek Agarwal
        2. OOZIE_1821-V2.patch
          2 kB
          Abhishek Agarwal

          Issue Links

            Activity

              People

              • Assignee:
                abhishek.agarwal Abhishek Agarwal
                Reporter:
                abhishek.agarwal Abhishek Agarwal
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: