Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: Coprocessors
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

      Use classloader which looks on HDFS.

      Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

      Not completely arbitrary code, should require implementation of an interface with callbacks for:

      • Open
      • Close
      • Split
      • Compact
      • (Multi)get and scanner next()
      • (Multi)put
      • (Multi)delete

      Add method to HTableInterface for invoking coprocessor methods and retrieving results.

      Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

      GitHub: https://github.com/trendmicro/hbase/tree/coprocessor

      Please see the latest attached package-info.html for updated description.

      1. asm-transformations.pdf
        76 kB
        Andrew Purtell
      2. HBASE-2001.patch.gz
        23 kB
        Andrew Purtell
      3. HBase-2001-final.patch
        161 kB
        Mingjie Lai
      4. HBASE-2001-RegionObserver.patch
        75 kB
        Andrew Purtell
      5. HBASE-2001-RegionObserver-2.patch
        85 kB
        Mingjie Lai
      6. packge-info.html
        13 kB
        Mingjie Lai
      7. packge-info.html
        13 kB
        Mingjie Lai
      8. packge-info.html
        12 kB
        Mingjie Lai

        Issue Links

          Activity

          Andrew Purtell created issue -
          Andrew Purtell made changes -
          Field Original Value New Value
          Link This issue depends upon HBASE-1936 [ HBASE-1936 ]
          Andrew Purtell made changes -
          Description "Support arbitrary code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also."

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.
          "Support arbitrary code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also."

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.
          Andrew Purtell made changes -
          Attachment asm-3.2-bin.zip [ 12425768 ]
          Andrew Purtell made changes -
          Attachment asm-transformations.pdf [ 12425769 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.HCoprocessor.java [ 12426312 ]
          Attachment org.apache.hadoop.hbase.HCoprocessor.pdf [ 12426313 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.HCoprocessor.java [ 12426341 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.HCoprocessor.java [ 12426312 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.HCoprocessor.pdf [ 12426313 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.HCoprocessor.java [ 12426341 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.coprocessor.Coprocessor.pdf [ 12427053 ]
          Attachment HBASE-2001.patch [ 12427054 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch [ 12427054 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch [ 12428486 ]
          Andrew Purtell made changes -
          Summary Coprocessors: Colocate arbitrary code with regions Coprocessors: Colocate user code with regions
          Description "Support arbitrary code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also."

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.
          Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch [ 12428486 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12428886 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12428886 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12431231 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12431231 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12434420 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12434420 ]
          Andrew Purtell made changes -
          Attachment org.apache.hadoop.hbase.coprocessor.Coprocessor.pdf [ 12427053 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435150 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435150 ]
          Andrew Purtell made changes -
          Link This issue depends upon HBASE-1936 [ HBASE-1936 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435791 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435791 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435835 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12435835 ]
          Andrew Purtell made changes -
          Attachment HBASE-2001.patch.gz [ 12436076 ]
          Andrew Purtell made changes -
          Description Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.
          Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

          GitHub: http://github.com/apurtell/hbase-coprocessor
          Andrew Purtell made changes -
          Attachment HBASE-2001-RegionObserver.patch [ 12445617 ]
          Andrew Purtell made changes -
          Link This issue is blocked by HBASE-2375 [ HBASE-2375 ]
          Eugene Koontz made changes -
          Link This issue is related to HBASE-2461 [ HBASE-2461 ]
          Mingjie Lai made changes -
          Attachment HBASE-2001-RegionObserver-2.patch [ 12453645 ]
          Andrew Purtell made changes -
          Link This issue is blocked by HBASE-2375 [ HBASE-2375 ]
          Andrew Purtell made changes -
          Link This issue is blocked by HBASE-2321 [ HBASE-2321 ]
          Andrew Purtell made changes -
          Attachment asm-3.2-bin.zip [ 12425768 ]
          Mingjie Lai made changes -
          Attachment packge-info.html [ 12455004 ]
          Andrew Purtell made changes -
          Link This issue relates to HBASE-1697 [ HBASE-1697 ]
          Andrew Purtell made changes -
          Link This issue blocks HBASE-3025 [ HBASE-3025 ]
          Andrew Purtell made changes -
          Assignee Andrew Purtell [ apurtell ] Mingjie Lai [ mingjielai ]
          Fix Version/s 0.90.0 [ 12313607 ]
          Mingjie Lai made changes -
          Attachment packge-info.html [ 12456164 ]
          Mingjie Lai made changes -
          Description Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

          GitHub: http://github.com/apurtell/hbase-coprocessor
          Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

          GitHub: http://github.com/mlai/hbase/tree/0.90_coprocessor
          stack made changes -
          Fix Version/s 0.92.0 [ 12314223 ]
          Fix Version/s 0.90.0 [ 12313607 ]
          Andrew Purtell made changes -
          Link This issue is related to HBASE-2357 [ HBASE-2357 ]
          Mingjie Lai made changes -
          Attachment packge-info.html [ 12459097 ]
          Mingjie Lai made changes -
          Description Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HRegionInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

          GitHub: http://github.com/mlai/hbase/tree/0.90_coprocessor
          Support user code that runs run next to each region in table. As regions split and move, coprocessor code should automatically move also.

          Use classloader which looks on HDFS.

          Associate a list of classes to load with each table. Put this in HRI so it inherits from table but can be changed on a per region basis (so then those region specific changes can inherited by daughters).

          Not completely arbitrary code, should require implementation of an interface with callbacks for:
          * Open
          * Close
          * Split
          * Compact
          * (Multi)get and scanner next()
          * (Multi)put
          * (Multi)delete

          Add method to HTableInterface for invoking coprocessor methods and retrieving results.

          Add methods in o.a.h.h.regionserver or subpackage which implement convenience functions for coprocessor methods and consistent/controlled access to internals: store access, threading, persistent and ephemeral state, scratch storage, etc.

          GitHub: https://github.com/trendmicro/hbase/tree/coprocessor

          Please see the latest attached package-info.html for updated description.
          Mingjie Lai made changes -
          Attachment HBase-2001-final.patch [ 12460068 ]
          Andrew Purtell made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Resolution Fixed [ 1 ]
          Todd Lipcon made changes -
          Component/s coprocessors [ 12314191 ]
          Gavin made changes -
          Link This issue blocks HBASE-3025 [ HBASE-3025 ]
          Gavin made changes -
          Link This issue is depended upon by HBASE-3025 [ HBASE-3025 ]

            People

            • Assignee:
              Mingjie Lai
              Reporter:
              Andrew Purtell
            • Votes:
              1 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development