Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-18169

Coprocessor fix and cleanup before 2.0.0 release

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha-1
    • Fix Version/s: 2.0.0-beta-1, 2.0.0
    • Component/s: Coprocessors
    • Labels:
      None
    • Release Note:
      Hide
      Refactor of Coprocessor API for hbase2. Purged methods that exposed too much of our internals. Other hooks were recast so they no longer took or returned internal classes; instead we pass Interfaces or read-only versions of implementations.

      Here is some overview doc on changes in hbase2 for Coprocessors including detail on why the change was made:
      https://github.com/apache/hbase/blob/branch-2.0/dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732.adoc

      More general background on why CPs had to change: https://docs.google.com/document/d/10cabwp_aR3OmpHVoeh544YLC3KwqMD9KiTIrHZAmfec/edit#
      Show
      Refactor of Coprocessor API for hbase2. Purged methods that exposed too much of our internals. Other hooks were recast so they no longer took or returned internal classes; instead we pass Interfaces or read-only versions of implementations. Here is some overview doc on changes in hbase2 for Coprocessors including detail on why the change was made: https://github.com/apache/hbase/blob/branch-2.0/dev-support/design-docs/Coprocessor_Design_Improvements-Use_composition_instead_of_inheritance-HBASE-17732.adoc More general background on why CPs had to change: https://docs.google.com/document/d/10cabwp_aR3OmpHVoeh544YLC3KwqMD9KiTIrHZAmfec/edit#

      Description

      As discussed in HBASE-18038. In RegionServerServices, Region and StoreFile interfaces we expose too many unnecessary methods. We need to find a way to not expose these methods to CP.

        Attachments

          Issue Links

          1.
          Region interface cleanup for CP expose Sub-task Resolved Anoop Sam John
          2.
          RegionServerServices Interface cleanup for CP expose Sub-task Resolved Anoop Sam John
          3.
          Mark StoreFileScanner/StoreFileReader as IA.LimitedPrivate(Phoenix) Sub-task Resolved Duo Zhang
          4.
          CompactionRequest should not be exposed to user directly Sub-task Resolved Duo Zhang
          5.
          Introduce a new constructor for StoreScanner for MOB Sub-task Resolved Duo Zhang
          6.
          Remove LegacyScanQueryMatcher Sub-task Resolved Duo Zhang
          7.
          Introduce new example and helper classes to tell CP users how to do filtering on scanners Sub-task Resolved Duo Zhang
          8.
          Remove bypass method in ObserverContext and implement the 'bypass' logic case by case Sub-task Resolved stack
          9.
          Cleanup bypass semantic in MasterCoprocessorHost Sub-task Resolved Heng Chen
          10.
          MasterServices needs a short-back-and-sides; pare-back exposure of internals and IA.Private classes Sub-task Resolved stack
          11.
          Remove deprecated methods in RegionObserver Sub-task Resolved Duo Zhang
          12.
          Remove deprecated methods in MasterObserver Sub-task Resolved Chia-Ping Tsai
          13.
          Remove the unused methods in RegionServerObserver Sub-task Resolved Chia-Ping Tsai
          14.
          We need to pass something like CompactionRequest in CP to give user some information about the compaction Sub-task Resolved Peter Somogyi
          15.
          Apply RegionInfo to MasterObserver/RegionObserver/WALObserver Sub-task Resolved Chia-Ping Tsai
          16.
          Use HStoreFile instead of StoreFile in our own code base and remove unnecessary methods in StoreFile interface Sub-task Resolved Duo Zhang
          17.
          Use HStore instead of Store in our own code base and remove unnecessary methods in Store interface Sub-task Resolved Duo Zhang
          18.
          Apply RegionInfo to code base Sub-task Resolved Chia-Ping Tsai
          19.
          Purge PB from BulkLoadObserver Sub-task Resolved stack
          20.
          Use Optional<T> return types when T can be null Sub-task Resolved Duo Zhang
          21.
          Allow CPs to request flush on Region and know the completion of the requested flush Sub-task Resolved Duo Zhang
          22.
          Provide Region#waitForFlushes API Sub-task Resolved Anoop Sam John
          23.
          Make ObserverContext an interface and remove private/testing methods Sub-task Resolved Appy
          24.
          Make a IA.LimitedPrivate interface for CellComparator Sub-task Resolved ramkrishna.s.vasudevan
          25.
          Remove the CompactionRequest parameter in preCompactSelection Sub-task Resolved Peter Somogyi
          26.
          Make *CoprocessorHost classes private Sub-task Resolved Appy
          27.
          Hide protobufs in GlobalQuotaSettings Sub-task Resolved Josh Elser
          28.
          Polish the compaction related CP hooks Sub-task Resolved Duo Zhang
          29.
          Remove the hooks in RegionObserver which are designed to construct a StoreScanner which is marked as IA.Private Sub-task Resolved Duo Zhang
          30.
          Introduce more examples to show how to intercept normal region operations Sub-task Resolved Josh Elser
          31.
          Allow CP users to change versions and TTL before opening StoreScanner Sub-task Resolved Duo Zhang
          32.
          Deprecate RegionObserver#postInstantiateDeleteTracker Sub-task Resolved Anoop Sam John
          33.
          RegionObserver#postCompactSelection Avoid passing shaded ImmutableList param Sub-task Resolved Anoop Sam John
          34.
          CP exposed Scanner types should not extend Shipper Sub-task Resolved Anoop Sam John
          35.
          Cleanup MasterObserver hooks which takes IA private params Sub-task Resolved stack
          36.
          Do not expose getHDFSBlockDistribution in StoreFile Sub-task Resolved Anoop Sam John
          37.
          Move methods that are for internal usage from CellUtil to Private util class Sub-task Resolved ramkrishna.s.vasudevan
          38.
          Do not wrap the original CompactionLifeCycleTracker when calling CP hooks Sub-task Resolved Duo Zhang
          39.
          Miscellaneous Observer cleanups Sub-task Resolved stack
          40.
          Make Tag IA.LimitedPrivate and expose for CPs Sub-task Resolved ramkrishna.s.vasudevan
          41.
          Add CP hooks in RegionObserver for in memory compaction Sub-task Resolved Duo Zhang
          42.
          Add the ClusterStatus hook and cleanup other hooks which can be replaced by ClusterStatus hook Sub-task Resolved Chia-Ping Tsai
          43.
          preCompact and preFlush can bypass by returning null scanner; shut it down Sub-task Resolved stack
          44.
          CoprocessorEnvironment should be exposed to CPs Sub-task Resolved Anoop Sam John
          45.
          The Configuration returned by CPEnv should be read-only. Sub-task Resolved stack
          46.
          Provide way for CPs to create short circuited connection with custom configurations Sub-task Resolved Anoop Sam John

            Activity

              People

              • Assignee:
                Apache9 Duo Zhang
                Reporter:
                Apache9 Duo Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: