Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18169

Coprocessor fix and cleanup before 2.0.0 release

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.0-alpha-1
    • 2.0.0-beta-1, 2.0.0
    • Coprocessors
    • None
    • Hide
      Refactor of Coprocessor API for hbase2. Purged methods that exposed too much of our internals. Other hooks were recast so they no longer took or returned internal classes; instead we pass Interfaces or read-only versions of implementations.

      Here is some overview doc on changes in hbase2 for Coprocessors including detail on why the change was made:
      https://github.com/apache/hbase/blob/branch-2.0/dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732.adoc

      More general background on why CPs had to change: https://docs.google.com/document/d/10cabwp_aR3OmpHVoeh544YLC3KwqMD9KiTIrHZAmfec/edit#
      Show
      Refactor of Coprocessor API for hbase2. Purged methods that exposed too much of our internals. Other hooks were recast so they no longer took or returned internal classes; instead we pass Interfaces or read-only versions of implementations. Here is some overview doc on changes in hbase2 for Coprocessors including detail on why the change was made: https://github.com/apache/hbase/blob/branch-2.0/dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732.adoc More general background on why CPs had to change: https://docs.google.com/document/d/10cabwp_aR3OmpHVoeh544YLC3KwqMD9KiTIrHZAmfec/edit#

    Description

      As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile interfaces we expose too many unnecessary methods. We need to find a way to not expose these methods to CP.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-16769 Deprecate/remove PB references from MasterObserver and RegionServerObserver

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-17732 Coprocessor Design Improvements

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HBASE-17312 [JDK8] Use default method for Observer Coprocessors

          • Major - Major loss of function.
          • Closed
          1.
          		 Region interface cleanup for CP expose Sub-task Closed Anoop Sam John
          2.
          		 RegionServerServices Interface cleanup for CP expose Sub-task Closed Anoop Sam John
          3.
          		 Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) Sub-task Closed Duo Zhang
          4.
          		 CompactionRequest should not be exposed to user directly Sub-task Closed Duo Zhang
          5.
          		 Introduce a new constructor for StoreScanner for MOB Sub-task Closed Duo Zhang
          6.
          		 Remove LegacyScanQueryMatcher Sub-task Closed Duo Zhang
          7.
          		 Introduce new example and helper classes to tell CP users how to do filtering on scanners Sub-task Closed Duo Zhang
          8.
          		 Remove bypass method in ObserverContext and implement the 'bypass' logic case by case Sub-task Closed Michael Stack
          9.
          		 Cleanup bypass semantic in MasterCoprocessorHost Sub-task Closed Heng Chen
          10.
          		 MasterServices needs a short-back-and-sides; pare-back exposure of internals and IA.Private classes Sub-task Closed Unassigned
          11.
          		 Remove deprecated methods in RegionObserver Sub-task Closed Duo Zhang
          12.
          		 Remove deprecated methods in MasterObserver Sub-task Closed Chia-Ping Tsai
          13.
          		 Remove the unused methods in RegionServerObserver Sub-task Closed Chia-Ping Tsai
          14.
          		 We need to pass something like CompactionRequest in CP to give user some information about the compaction Sub-task Closed Peter Somogyi
          15.
          		 Apply RegionInfo to MasterObserver/RegionObserver/WALObserver Sub-task Closed Chia-Ping Tsai
          16.
          		 Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface Sub-task Closed Duo Zhang
          17.
          		 Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface Sub-task Closed Duo Zhang
          18.
          		 Apply RegionInfo to code base Sub-task Closed Chia-Ping Tsai
          19.
          		 Purge PB from BulkLoadObserver Sub-task Closed Michael Stack
          20.
          		 Use Optional<T> return types when T can be null Sub-task Closed Duo Zhang
          21.
          		 Allow CPs to request flush on Region and know the completion of the requested flush Sub-task Closed Duo Zhang
          22.
          		 Provide Region#waitForFlushes API Sub-task Closed Anoop Sam John
          23.
          		 Make ObserverContext an interface and remove private/testing methods Sub-task Closed Apekshit Sharma
          24.
          		 Make a IA.LimitedPrivate interface for CellComparator Sub-task Closed ramkrishna.s.vasudevan
          25.
          		 Remove the CompactionRequest parameter in preCompactSelection Sub-task Closed Peter Somogyi
          26.
          		 Make *CoprocessorHost classes private Sub-task Closed Apekshit Sharma
          27.
          		 Hide protobufs in GlobalQuotaSettings Sub-task Closed Josh Elser
          28.
          		 Polish the compaction related CP hooks Sub-task Closed Duo Zhang
          29.
          		 Remove the hooks in RegionObserver which are designed to construct a StoreScanner which is marked as IA.Private Sub-task Closed Duo Zhang
          30.
          		 Introduce more examples to show how to intercept normal region operations Sub-task Closed Josh Elser
          31.
          		 Allow CP users to change versions and TTL before opening StoreScanner Sub-task Closed Duo Zhang
          32.
          		 Deprecate RegionObserver#postInstantiateDeleteTracker Sub-task Closed Anoop Sam John
          33.
          		 RegionObserver#postCompactSelection Avoid passing shaded ImmutableList param Sub-task Closed Anoop Sam John
          34.
          		 CP exposed Scanner types should not extend Shipper Sub-task Closed Anoop Sam John
          35.
          		 Cleanup MasterObserver hooks which takes IA private params Sub-task Closed Michael Stack
          36.
          		 Do not expose getHDFSBlockDistribution in StoreFile Sub-task Closed Anoop Sam John
          37.
          		 Move methods that are for internal usage from CellUtil to Private util class Sub-task Closed ramkrishna.s.vasudevan
          38.
          		 Do not wrap the original CompactionLifeCycleTracker when calling CP hooks Sub-task Closed Duo Zhang
          39.
          		 Miscellaneous Observer cleanups Sub-task Closed Michael Stack
          40.
          		 Make Tag IA.LimitedPrivate and expose for CPs Sub-task Closed ramkrishna.s.vasudevan
          41.
          		 Add CP hooks in RegionObserver for in memory compaction Sub-task Closed Duo Zhang
          42.
          		 Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook Sub-task Closed Chia-Ping Tsai
          43.
          		 preCompact and preFlush can bypass by returning null scanner; shut it down Sub-task Closed Michael Stack
          44.
          		 CoprocessorEnvironment should be exposed to CPs Sub-task Closed Anoop Sam John
          45.
          		 The Configuration returned by CPEnv should be read-only. Sub-task Closed Michael Stack
          46.
          		 Provide way for CPs to create short circuited connection with custom configurations Sub-task Closed Anoop Sam John

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: