HBase
  1. HBase
  2. HBASE-79

[hbase] When HBase needs to be migrated, it should display a message on stdout, not just in the logs

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.1.0, 0.2.0
    • Fix Version/s: 0.1.0, 0.2.0
    • Component/s: util
    • Labels:
      None

      Description

      When you upgrade your HBase code version, there is occasionally the need to migrate the underlying data store to a new version. However, if you are unaware of this need, then you'll be very confused by what happens when you restart HBase. Using start-hbase.sh, you get messages indicating that the master and regionservers started as expected. However, in reality, it will have tried to start and failed due to a version mismatch. This information is displayed in the logs, but you won't know that until you go log diving.

      Instead, let's have the start-hbase.sh script do a check to see if the version number is wrong itself. That way, if it fails, it can write messages about startup failure to the console instead of to the logs. This will make new admins much happier.

      1. 79.patch
        0.9 kB
        stack

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        19d 20h 1m 1 stack 19/Feb/08 19:38
        Resolved Resolved Reopened Reopened
        16d 2h 6m 1 stack 06/Mar/08 21:45
        Reopened Reopened Resolved Resolved
        21h 49m 1 Jim Kellerman 07/Mar/08 19:34
        Resolved Resolved Closed Closed
        168d 1h 38m 1 Jim Kellerman 22/Aug/08 22:13
        Jim Kellerman made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Jim Kellerman made changes -
        Fix Version/s 0.2.0 [ 12312955 ]
        Affects Version/s 0.2.0 [ 12312955 ]
        Jim Kellerman made changes -
        Fix Version/s 0.1.0 [ 12312954 ]
        Resolution Fixed [ 1 ]
        Status Reopened [ 4 ] Resolved [ 5 ]
        Fix Version/s 0.2.0 [ 12312955 ]
        Hide
        Jim Kellerman added a comment -

        Backported same patch from trunk to 0.1.0. Committed.

        Show
        Jim Kellerman added a comment - Backported same patch from trunk to 0.1.0. Committed.
        Jim Kellerman made changes -
        Assignee Jim Kellerman [ jimk ]
        stack made changes -
        Priority Major [ 3 ] Blocker [ 1 ]
        Affects Version/s 0.1.0 [ 12312954 ]
        stack made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        stack added a comment -

        Lets backport to 0.1.0 release.

        Show
        stack added a comment - Lets backport to 0.1.0 release.
        stack made changes -
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Hide
        stack added a comment -

        Committed w/ Bryan's better wording.

        Show
        stack added a comment - Committed w/ Bryan's better wording.
        Hide
        Bryan Duxbury added a comment -

        Patch looks good, maybe change the message though? That message seems to indicate I have to upgrade HDFS.

        How about "The HBase data files stored in HDFS are from an earlier version of HBase. You need to run 'bin/hbase migrate' to bring your installation up-to-date." ?

        Show
        Bryan Duxbury added a comment - Patch looks good, maybe change the message though? That message seems to indicate I have to upgrade HDFS. How about "The HBase data files stored in HDFS are from an earlier version of HBase. You need to run 'bin/hbase migrate' to bring your installation up-to-date." ?
        stack made changes -
        Attachment 79.patch [ 12375956 ]
        Hide
        stack added a comment -

        Patch that outputs this when you try to start a master on a system that needs migration:

        durruti:~/Documents/checkouts/hbase/trunk stack$ ./bin/start-hbase.sh 
        starting master, logging to /Users/stack/Documents/checkouts/hbase/trunk/bin/../logs/hbase-stack-master-durruti.desk.hq.powerset.com.out
        WARNING! File system needs upgrade. Run the '${HBASE_HOME}/bin/hbase migrate' script. Master shutting down...
        
        Show
        stack added a comment - Patch that outputs this when you try to start a master on a system that needs migration: durruti:~/Documents/checkouts/hbase/trunk stack$ ./bin/start-hbase.sh starting master, logging to /Users/stack/Documents/checkouts/hbase/trunk/bin/../logs/hbase-stack-master-durruti.desk.hq.powerset.com.out WARNING! File system needs upgrade. Run the '${HBASE_HOME}/bin/hbase migrate' script. Master shutting down...
        Hide
        stack added a comment -

        This might not be that hard to do. I've just been messing in scripts. I noticed that it does a 'head hbase-master*out' after startup before exiting. If the migration message was put on STDOUT/ERR, rather than logged (or put in both places), I think that should be all thats needed to make this work.

        Show
        stack added a comment - This might not be that hard to do. I've just been messing in scripts. I noticed that it does a 'head hbase-master*out' after startup before exiting. If the migration message was put on STDOUT/ERR, rather than logged (or put in both places), I think that should be all thats needed to make this work.
        Bryan Duxbury made changes -
        Fix Version/s 0.2.0 [ 12312955 ]
        Bryan Duxbury made changes -
        Component/s util [ 12312144 ]
        Owen O'Malley made changes -
        Fix Version/s 0.17.0 [ 12312913 ]
        Key HADOOP-2745 HBASE-79
        Component/s contrib/hbase [ 12311752 ]
        Project Hadoop Core [ 12310240 ] Hadoop HBase [ 12310753 ]
        Bryan Duxbury made changes -
        Fix Version/s 0.17.0 [ 12312913 ]
        Bryan Duxbury made changes -
        Field Original Value New Value
        Description When you upgrade your HBase code version, there is occasionally the need to migrate the underlying data store to a new version. However, if you are unaware of this need, then you'll be very confused by what happens when you restart HBase. Using start-hbase.sh, you get messages indicating that the master and regionservers started as expected. However, in reality, it will have tried to start and failed due to a version mismatch. This information is displayed in the logs, but you won't know that until you go log diving.

        Instead, let's have the start-hbase.sh script do a check to see if the version number is wrong itself. That way, if it fails, it can write messages about startup failure to the console instead of to the logs. This will make new admins much happier.
        Bryan Duxbury created issue -

          People

          • Assignee:
            Jim Kellerman
            Reporter:
            Bryan Duxbury
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development