[HBASE-15430] Failed taking snapshot - Manifest proto-message too large

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.98.11
• Fix Version/s: 1.3.0, 1.2.1, 1.1.4, 0.98.18, 2.0.0
• Component/s: snapshots
• Release Note: Failed taking snapshot - Manifest proto-message too large. add property ("snapshot.manifest.size.limit") to change max size of proto-message
• Tags: snapshot, protocol buffer
• Flags: Patch
• Assignee: JunHo Cho
• Reporter: JunHo Cho
• Created: 09/Mar/16 01:10
• Updated: 17/Mar/16 18:31
• Resolved: 16/Mar/16 16:35

Description

the size of a protobuf message is 64MB (default). but the size of snapshot meta is over 64MB.

Caused by: com.google.protobuf.InvalidProtocolBufferException via Failed taking snapshot

{ ss=snapshot_xxx table=xxx type=FLUSH }

due to exception:Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.:com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.
at org.apache.hadoop.hbase.errorhandling.ForeignExceptionDispatcher.rethrowException(ForeignExceptionDispatcher.java:83)
at org.apache.hadoop.hbase.master.snapshot.TakeSnapshotHandler.rethrowExceptionIfFailed(TakeSnapshotHandler.java:307)
at org.apache.hadoop.hbase.master.snapshot.SnapshotManager.isSnapshotDone(SnapshotManager.java:341)
... 10 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message was too large. May be malicious. Use CodedInputStream.setSizeLimit() to increase the size limit.
at com.google.protobuf.InvalidProtocolBufferException.sizeLimitExceeded(InvalidProtocolBufferException.java:110)
at com.google.protobuf.CodedInputStream.refillBuffer(CodedInputStream.java:755)
at com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:811)
at com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329)
at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.<init>(HBaseProtos.java:3767)
at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo.<init>(HBaseProtos.java:3699)
at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3815)
at org.apache.hadoop.hbase.protobuf.generated.HBaseProtos$RegionInfo$1.parsePartialFrom(HBaseProtos.java:3810)
at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.<init>(SnapshotProtos.java:1152)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest.<init>(SnapshotProtos.java:1094)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1201)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotRegionManifest$1.parsePartialFrom(SnapshotProtos.java:1196)
at com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:309)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.<init>(SnapshotProtos.java:3858)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.<init>(SnapshotProtos.java:3792)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3894)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest$1.parsePartialFrom(SnapshotProtos.java:3889)
at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:217)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:223)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:49)
at org.apache.hadoop.hbase.protobuf.generated.SnapshotProtos$SnapshotDataManifest.parseFrom(SnapshotProtos.java:4094)
at org.apache.hadoop.hbase.snapshot.SnapshotManifest.readDataManifest(SnapshotManifest.java:433)
at org.apache.hadoop.hbase.snapshot.SnapshotManifest.load(SnapshotManifest.java:273)
at org.apache.hadoop.hbase.snapshot.SnapshotManifest.open(SnapshotManifest.java:119)
at org.apache.hadoop.hbase.master.snapshot.MasterSnapshotVerifier.verifySnapshot(MasterSnapshotVerifier.java:106