Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1960

Master should wait for DFS to come up when creating hbase.version

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      This has been addressed as part of HBASE-5003 and others.

      Description

      The master does not wait for DFS to come up in the circumstance where the DFS master is started for the first time after format and no datanodes have been started yet.

      2009-11-07 11:47:28,115 INFO org.apache.hadoop.hbase.master.HMaster: vmName=Java HotSpot(TM) 64-Bit Server VM, vmVendor=Sun Microsystems Inc., vmVersion=14.2-b01
      2009-11-07 11:47:28,116 INFO org.apache.hadoop.hbase.master.HMaster: vmInputArguments=[-Xmx1000m, -XX:+HeapDumpOnOutOfMemoryError, -XX:+UseConcMarkSweepGC, -XX:+CMSIncrementalMode, -Dhbase.log.dir=/mnt/hbase/logs, -Dhbase.log.file=hbase-root-master-ip-10-242-15-159.log, -Dhbase.home.dir=/usr/local/hbase-0.20.1/bin/.., -Dhbase.id.str=root, -Dhbase.root.logger=INFO,DRFA, -Djava.library.path=/usr/local/hbase-0.20.1/bin/../lib/native/Linux-amd64-64]
      2009-11-07 11:47:28,247 INFO org.apache.hadoop.hbase.master.HMaster: My address is ip-10-242-15-159.ec2.internal:60000
      2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /hbase/hbase.version could only be replicated to 0 nodes, instead of 1
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
      
      [...]
      
      2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes == null
      2009-11-07 11:47:28,728 WARN org.apache.hadoop.hdfs.DFSClient: Could not get block locations. Source file "/hbase/hbase.version" - Aborting...
      2009-11-07 11:47:28,729 FATAL org.apache.hadoop.hbase.master.HMaster: Not starting HMaster because:
      org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /hbase/hbase.version could only be replicated to 0 nodes, instead of 1
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1267)
      	at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:422)
      

      Should probably sleep and retry the write a few times.

        Attachments

        1. HBASE-1960.patch
          1 kB
          Andrew Kyle Purtell
        2. HBASE-1960-redux.patch
          5 kB
          Andrew Kyle Purtell

          Issue Links

            Activity

              People

              • Assignee:
                apurtell Andrew Kyle Purtell
                Reporter:
                apurtell Andrew Kyle Purtell
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: