[HBASE-7360] Snapshot 0.94 Backport

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.94.3
• Fix Version/s: 0.94.6
• Component/s: snapshots
• Hadoop Flags: Reviewed
• Assignee: Matteo Bertozzi
• Reporter: Matteo Bertozzi
• Created: 14/Dec/12 23:37
• Updated: 23/Mar/13 04:53
• Resolved: 05/Mar/13 22:53

Description

Backport snapshot code to 0.94

The main changes needed to backport the snapshot are related to the protobuf vs writable rpc.

Offline Snapshot

• HBASE-6610 HFileLink: Hardlink alternative for snapshot restore
• HBASE-6765 Take a Snapshot interface
  • This patch is significantly different because of the RPC changes between 0.94/0.96
• HBASE-6571 Generic multi-thread/cross-process error handling framework
• HBASE-6353 Snapshots shell
• HBASE-7107 Snapshot References Utils (FileSystem Visitor)
• HBASE-6836 Offline snapshots
• HBASE-6865 Snapshot File Cleaners
• HBASE-6777 Snapshot Restore Interface
  • This patch is significantly different because of the RPC changes between 0.94/0.96
• HBASE-6230 Clone/Restore Snapshots
• HBASE-6802 Export Snapshot
• HBASE-7240 Cleanup old snapshots on start
• HBASE-7174 Refactor snapshot file cleaner cache to use Snapshot
• HBASE-7367 Snapshot coprocessor and ACL security
• HBASE-7311 Add snapshot information to hbase master webui
  • This patch is significantly different because of the RPC changes between 0.94/0.96
• HBASE-7418 HFileLink flaky test
• HBASE-7420 TestSnapshotExceptionSnare and TestWALReferenceTask missing test category annotation and failing TestCheckTestClasses
• HBASE-7206 ForeignException framework v2 (simplifies and replaces HBASE-6571)
• HBASE-7430 TestSnapshotDescriptionUtils break compaction/scanner tests (EnvironmentEdge issue)
• HBASE-7436 Improve stack trace info dumped by ForeignExceptionSnare#rethrowException
• HBASE-7339 Splitting an HFileLink causes region servers to go down.
• HBASE-7439 HFileLink should not use the configuration from the Filesystem
• HBASE-7452 Change ForeignExceptionListener#receive(String, FE) to only be #receive(FE)
• HBASE-7208 Transition Offline Snapshots to ForeignExceptions and Refactor for merge
• HBASE-7207 Consolidate snapshot related classes into fewer packages
• HBASE-7400 ExportSnapshot mapper closes the FileSystem
• HBASE-7352 clone operation from HBaseAdmin can hang forever
• HBASE-7294 Check for snapshot file cleaners on start
• HBASE-7354 Snapshot Info/Debug Tool
• HBASE-7423 HFileArchiver should not use the configuration from the Filesystem
• HBASE-7453 HBASE-7423 snapshot followup
• HBASE-7212 Globally Barriered Procedure Mechanism
• HBASE-6864 Online snapshots scaffolding
• HBASE-7321 Simple Flush Snapshot
• HBASE-7496 TestZKProcedure fails interrmittently.
• HBASE-7484 Fix Restore with schema changes
• HBASE-7419 revisit hfilelink file name format
• HBASE-7467 CleanerChore checkAndDeleteDirectory not deleting empty directories
• HBASE-7214 CleanerChore logs too much, so much so it obscures all else that is going on
• HBASE-7523 Snapshot attempt with the name of a previously taken snapshot fails sometimes.
• HBASE-7480 Explicit message for not allowed snapshot on meta tables
• HBASE-7537 .regioninfo not created by createHRegion()
• HBASE-7535 Fix restore reference files
• HBASE-7552 Pass bufferSize param to FileLinkInputStream constructor within FileLink.open method, and remove unnecessary import packages.
• HBASE-7501 Introduce MetaEditor method that both adds and deletes rows in .META. table
• HBASE-7365/HBASE-7389 Safer table creation and deletion using .tmp dir
• HBASE-7547 Fix findbugs warnings in snapshot classes
• HBASE-7548 Fix javadoc warnings in snapshot classes
• HBASE-7538 Improve snapshot related error and exception messages
• HBASE-7536 Add test that confirms that multiple concurrent snapshot requests are rejected
• HBASE-7583 Fixes and cleanups
• HBASE-7604 Remove duplicated code from HFileLink
• HBASE-7616 NPE in ZKProcedure.nodeCreated
• HBASE-7625 Remove duplicated logFSTree() from TestRestoreFlushSnapshotFromClient
• HBASE-7627 UnsupportedOperationException in CatalogJanitor thread
• HBASE-7622 Add table descriptor verification after snapshot restore
• HBASE-7651 RegionServerSnapshotManager fails with CancellationException if previous snapshot fails in per region task
• HBASE-7666 More logging improvements in online snapshots code
• HBASE-7689 CloneTableHandler notify completion too early
• HBASE-7699 Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
• HBASE-7703 Eventually all online snapshots failing due to Timeout at same regionserver.
• HBASE-7720 Improve logging messages of failed snapshot attempts.
• HBASE-7795 Race in the Restore Archiving
• HBASE-7788 Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
• HBASE-7858 cleanup before merging snapshots branch to trunk
• HBASE-7889 Fix javadoc warnings in snapshot classes
• HBASE-7911 Remove duplicated code from CreateTableHandler
• HBASE-7969 Rename HBaseAdmin#getCompletedSnapshots as HBaseAdmin#listSnapshots
• HBASE-7299 TestMultiParallel fails intermittently in trunk builds