Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-118

Namenode does not always clean up pendingCreates

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.1.0
    • Fix Version/s: 0.2.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      195 node cluster running redhat linux (2.6 kernel)

      Description

      In some failure modes, the pending creates list is not cleaned up and prevents that file from ever being created.

      When I try to create the file after the first job was killed (hours previously), I get:

      060404 084619 Cannot start file because pendingCreates is non-null. src=/user/oom/rand/part000118
      060404 084619 Server handler 0 on 8020 call error: java.io.IOException: Cannot create file /user/oom/rand/part000118 on client DFSClient_-1656137458
      java.io.IOException: Cannot create file /user/oom/rand/part000118 on client DFSClient_-1656137458
      at org.apache.hadoop.dfs.NameNode.create(NameNode.java:147)
      at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:237)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:216)

        Attachments

        1. release-create-locks.patch
          16 kB
          Owen O'Malley

          Activity

            People

            • Assignee:
              omalley Owen O'Malley
              Reporter:
              omalley Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: