[HBASE-18169] Coprocessor fix and cleanup before 2.0.0 release

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha-1
• Fix Version/s: 2.0.0-beta-1, 2.0.0
• Component/s: Coprocessors

Release Note

Refactor of Coprocessor API for hbase2. Purged methods that exposed too much of our internals. Other hooks were recast so they no longer took or returned internal classes; instead we pass Interfaces or read-only versions of implementations.

Here is some overview doc on changes in hbase2 for Coprocessors including detail on why the change was made:
https://github.com/apache/hbase/blob/branch-2.0/dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732.adoc

More general background on why CPs had to change: https://docs.google.com/document/d/10cabwp_aR3OmpHVoeh544YLC3KwqMD9KiTIrHZAmfec/edit#

• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 06/Jun/17 10:17
• Updated: 14/Jun/18 01:19
• Resolved: 07/Dec/17 20:53

Description

As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile interfaces we expose too many unnecessary methods. We need to find a way to not expose these methods to CP.

Issue Links

• is related to: HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver
• is related to: HBASE-17732 Coprocessor Design Improvements
• relates to: HBASE-17312 [JDK8] Use default method for Observer Coprocessors

Sub-Tasks

• Region interface cleanup for CP expose: Resolved
• RegionServerServices Interface cleanup for CP expose: Resolved
• Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix): Resolved
• CompactionRequest should not be exposed to user directly: Resolved
• Introduce a new constructor for StoreScanner for MOB: Resolved
• Remove LegacyScanQueryMatcher: Resolved
• Introduce new example and helper classes to tell CP users how to do filtering on scanners: Resolved
• Remove bypass method in ObserverContext and implement the 'bypass' logic case by case: Resolved
• Cleanup bypass semantic in MasterCoprocessorHost: Resolved
• MasterServices needs a short-back-and-sides; pare-back exposure of internals and IA.Private classes: Resolved
• Remove deprecated methods in RegionObserver: Resolved
• Remove deprecated methods in MasterObserver: Resolved
• Remove the unused methods in RegionServerObserver: Resolved
• We need to pass something like CompactionRequest in CP to give user some information about the compaction: Resolved
• Apply RegionInfo to MasterObserver/RegionObserver/WALObserver: Resolved
• Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface: Resolved
• Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface: Resolved
• Apply RegionInfo to code base: Resolved
• Purge PB from BulkLoadObserver: Resolved
• Use Optional<T> return types when T can be null: Resolved
• Allow CPs to request flush on Region and know the completion of the requested flush: Resolved
• Provide Region#waitForFlushes API: Resolved
• Make ObserverContext an interface and remove private/testing methods: Resolved
• Make a IA.LimitedPrivate interface for CellComparator: Resolved
• Remove the CompactionRequest parameter in preCompactSelection: Resolved
• Make *CoprocessorHost classes private: Resolved
• Hide protobufs in GlobalQuotaSettings: Resolved
• Polish the compaction related CP hooks: Resolved
• Remove the hooks in RegionObserver which are designed to construct a StoreScanner which is marked as IA.Private: Resolved
• Introduce more examples to show how to intercept normal region operations: Resolved
• Allow CP users to change versions and TTL before opening StoreScanner: Resolved
• Deprecate RegionObserver#postInstantiateDeleteTracker: Resolved
• RegionObserver#postCompactSelection Avoid passing shaded ImmutableList param: Resolved
• CP exposed Scanner types should not extend Shipper: Resolved
• Cleanup MasterObserver hooks which takes IA private params: Resolved
• Do not expose getHDFSBlockDistribution in StoreFile: Resolved
• Move methods that are for internal usage from CellUtil to Private util class: Resolved
• Do not wrap the original CompactionLifeCycleTracker when calling CP hooks: Resolved
• Miscellaneous Observer cleanups: Resolved
• Make Tag IA.LimitedPrivate and expose for CPs: Resolved
• Add CP hooks in RegionObserver for in memory compaction: Resolved
• Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook: Resolved
• preCompact and preFlush can bypass by returning null scanner; shut it down: Resolved
• CoprocessorEnvironment should be exposed to CPs: Resolved
• The Configuration returned by CPEnv should be read-only.: Resolved
• Provide way for CPs to create short circuited connection with custom configurations: Resolved