Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3415

During NameNode starting up, it may pick wrong storage directory inspector when the layout versions of the storage directories are different

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha, 3.0.0
    • Fix Version/s: 2.0.2-alpha
    • Component/s: namenode
    • Labels:
      None
    • Environment:

      Suse linux + jdk 1.6

      Description

      Scenario:
      =========
      start Namenode and datanode by configuring three storage dir's for namenode
      write 10 files
      edit version file of one of the storage dir and give layout version as 123 which different with default(-40).
      Stop namenode
      start Namenode.

      Then I am getting follwong exception...

      2012-05-13 19:01:41,483 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
      java.lang.NullPointerException
      	at org.apache.hadoop.hdfs.server.namenode.NNStorage.getStorageFile(NNStorage.java:686)
      	at org.apache.hadoop.hdfs.server.namenode.FSImagePreTransactionalStorageInspector.getEditsInStorageDir(FSImagePreTransactionalStorageInspector.java:243)
      	at org.apache.hadoop.hdfs.server.namenode.FSImagePreTransactionalStorageInspector.getLatestEditsFiles(FSImagePreTransactionalStorageInspector.java:261)
      	at org.apache.hadoop.hdfs.server.namenode.FSImagePreTransactionalStorageInspector.getEditLogStreams(FSImagePreTransactionalStorageInspector.java:276)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:596)
      	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:247)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:498)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:390)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:354)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:368)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:402)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:564)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:545)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1093)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1151)
      2012-05-13 19:01:41,485 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: 
      
      
      1. HDFS-3415.patch
        2 kB
        Brandon Li
      2. HDFS-3415.patch
        3 kB
        Brandon Li
      3. HDFS-3415.patch
        4 kB
        Brandon Li
      4. HDFS-3415.patch
        4 kB
        Brandon Li

        Issue Links

          Activity

          Arun C Murthy made changes -
          Fix Version/s 2.0.2-alpha [ 12322472 ]
          Fix Version/s 2.1.0-alpha [ 12321440 ]
          Brandon Li made changes -
          Link This issue relates to HDFS-3629 [ HDFS-3629 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 2.0.1 [ 12321440 ]
          Resolution Fixed [ 1 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Brandon Li made changes -
          Attachment HDFS-3415.patch [ 12528528 ]
          Brandon Li made changes -
          Attachment HDFS-3415.patch [ 12528471 ]
          Brandon Li made changes -
          Attachment HDFS-3415.patch [ 12528188 ]
          Brandon Li made changes -
          Summary NameNode is getting shutdown by throwing nullpointer exception when one of the layout version is different with others(Multiple storage dirs are configured) During NameNode starting up, it may pick wrong storage directory inspector when the layout versions of the storage directories are different
          Brandon Li made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Brandon Li made changes -
          Attachment HDFS-3415.patch [ 12527967 ]
          Brandon Li made changes -
          Assignee Brandon Li [ brandonli ]
          Brahma Reddy Battula made changes -
          Field Original Value New Value
          Fix Version/s 2.0.0 [ 12320353 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Brahma Reddy Battula created issue -

            People

            • Assignee:
              Brandon Li
              Reporter:
              Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development