[HBASE-4388] Second start after migration from 90 to trunk crashes

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.92.0
• Fix Version/s: 0.92.0
• Component/s: master, migration
• Hadoop Flags: Reviewed
• Assignee: Michael Stack
• Reporter: Todd Lipcon
• Created: 13/Sep/11 19:30
• Updated: 20/Nov/15 11:55
• Resolved: 27/Oct/11 21:07

Description

I started a trunk cluster to upgrade from 90, inserted a ton of data, then did a clean shutdown. When I started again, I got the following exception:

11/09/13 12:29:09 INFO master.HMaster: Meta has HRI with HTDs. Updating meta now.
11/09/13 12:29:09 FATAL master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NegativeArraySizeException: -102
at org.apache.hadoop.hbase.util.Bytes.readByteArray(Bytes.java:147)
at org.apache.hadoop.hbase.HTableDescriptor.readFields(HTableDescriptor.java:606)
at org.apache.hadoop.hbase.migration.HRegionInfo090x.readFields(HRegionInfo090x.java:641)
at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:133)
at org.apache.hadoop.hbase.util.Writables.getWritable(Writables.java:103)
at org.apache.hadoop.hbase.util.Writables.getHRegionInfoForMigration(Writables.java:228)
at org.apache.hadoop.hbase.catalog.MetaEditor.getHRegionInfoForMigration(MetaEditor.java:350)
at org.apache.hadoop.hbase.catalog.MetaEditor$1.visit(MetaEditor.java:273)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:633)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:255)
at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:235)
at org.apache.hadoop.hbase.catalog.MetaEditor.updateMetaWithNewRegionInfo(MetaEditor.java:284)
at org.apache.hadoop.hbase.catalog.MetaEditor.migrateRootAndMeta(MetaEditor.java:298)
at org.apache.hadoop.hbase.master.HMaster.updateMetaWithNewHRI(HMaster.java:529)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:472)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:309)

Issue Links

• is broken by: HBASE-3970 Address HMaster crash/failure half way through meta migration