Details

    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Hide
      We are moving from Inheritence
      - Observer *is* Coprocessor
      - FooService *is* CoprocessorService
      To Composition
      - Coprocessor *has* Observer
      - Coprocessor *has* Service
      ------------------------------------------------------
      Summary
      ------------------------------------------------------
      - Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,
        WALCoprocessor
      - These new *Coprocessor interfaces have a get*Observer() function for each observer type
        supported by them.
      - Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will
        get it from the base interface.
      - Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its
        own trickery.
      - CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.
        Deleted if can, else marked @VisibleForTesting.
      ------------------------------------------------------
      Backward Compatibility
      ------------------------------------------------------
      - Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees).
      - Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers).
      - Old coprocessors implementing CoprocessorService (for master/region host)
        /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0.
      - Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService
      - Note that if a coprocessor implements both observer and service in same class, its service
        component will continue to work but it's observer component won't work.
      Show
      We are moving from Inheritence - Observer *is* Coprocessor - FooService *is* CoprocessorService To Composition - Coprocessor *has* Observer - Coprocessor *has* Service ------------------------------------------------------ Summary ------------------------------------------------------ - Adds four new interfaces - MasterCoprocessor, RegionCoprocessor, RegionServierCoprocessor,   WALCoprocessor - These new *Coprocessor interfaces have a get*Observer() function for each observer type   supported by them. - Added Coprocessor#getService() to base interface. All extending *Coprocessor interfaces will   get it from the base interface. - Added BulkLoadObserver hooks to RegionCoprocessorHost instad of SecureBulkLoadManager doing its   own trickery. - CoprocessorHost#find*() fuctions: Too many testing hooks digging into CP internals.   Deleted if can, else marked @VisibleForTesting. ------------------------------------------------------ Backward Compatibility ------------------------------------------------------ - Old coprocessors implementing *Observer won't get loaded (no backward compatibility guarantees). - Third party coprocessors only implementing Coprocessor will not get loaded (just like Observers). - Old coprocessors implementing CoprocessorService (for master/region host)   /SingletonCoprocessorService (for RegionServer host) will continue to work with 2.0. - Added test to ensure backward compatibility of CoprocessorService/SingletonCoprocessorService - Note that if a coprocessor implements both observer and service in same class, its service   component will continue to work but it's observer component won't work.

      Description

      The two main changes are:

      • Adding template for coprocessor type to CoprocessorEnvironment i.e. interface CoprocessorEnvironment<C extends Coprocessor>
        • Enables us to load only relevant coprocessors in hosts. Right now each type of host loads all types of coprocs and it's only during execOperation that it checks if the coproc is of correct type i.e. XCoprocessorHost will load XObserver, YObserver, and all others, and will check in execOperation if coproc instanceOf XObserver and ignore the rest.
        • Allow sharing of a bunch functions/classes which are currently duplicated in each host. For eg. CoprocessorOperations, CoprocessorOperationWithResult, execOperations().
      • Introduce 4 coprocessor classes and use composition between these new classes and and old observers
        • The real gold here is, moving forward, we'll be able to break down giant everything-in-one observers (masterobserver has 100+ functions) into smaller, more focused observers. These smaller observer can then have different compat guarantees!!

      Here's a more detailed design doc: https://docs.google.com/document/d/1mPkM1CRRvBMZL4dBQzrus8obyvNnHhR5it2yyhiFXTg/edit?usp=sharing

        Attachments

        1. HBASE-17732.master.014.patch
          541 kB
          Appy
        2. HBASE-17732.master.013.patch
          541 kB
          Appy
        3. HBASE-17732.master.012.patch
          547 kB
          Appy
        4. HBASE-17732.master.011.patch
          545 kB
          Appy
        5. HBASE-17732.master.010.patch
          478 kB
          Appy
        6. HBASE-17732.master.009.patch
          476 kB
          Appy
        7. HBASE-17732.master.008.patch
          475 kB
          Appy
        8. HBASE-17732.master.007.patch
          475 kB
          Appy
        9. HBASE-17732.master.006.patch
          378 kB
          Appy
        10. HBASE-17732.master.005.patch
          300 kB
          Appy
        11. HBASE-17732.master.004.patch
          388 kB
          Appy
        12. HBASE-17732.master.003.patch
          307 kB
          Appy
        13. HBASE-17732.master.002.patch
          97 kB
          Appy
        14. HBASE-17732.master.001.patch
          107 kB
          Appy

          Issue Links

            Activity

              People

              • Assignee:
                appy Appy
                Reporter:
                appy Appy
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: