Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-13992

Integrate SparkOnHBase into HBase

    XMLWordPrintableJSON

Details

    • Hide
      This release includes initial support for running Spark against HBase with a richer feature set than was previously possible with MapReduce bindings:

      * Support for Spark and Spark Streaming against Spark 2.1.1
      * RDD/DStream formation from scan operations
      * convenience methods for interacting with HBase from an HBase backed RDD / DStream instance
      * examples in both the Spark Java API and Spark Scala API
      * support for running against a secure HBase cluster
      Show
      This release includes initial support for running Spark against HBase with a richer feature set than was previously possible with MapReduce bindings: * Support for Spark and Spark Streaming against Spark 2.1.1 * RDD/DStream formation from scan operations * convenience methods for interacting with HBase from an HBase backed RDD / DStream instance * examples in both the Spark Java API and Spark Scala API * support for running against a secure HBase cluster

    Description

      This Jira is to ask if SparkOnHBase can find a home in side HBase core.

      Here is the github:
      https://github.com/cloudera-labs/SparkOnHBase

      I am the core author of this project and the license is Apache 2.0

      A blog explaining this project is here
      http://blog.cloudera.com/blog/2014/12/new-in-cloudera-labs-sparkonhbase/

      A spark Streaming example is here
      http://blog.cloudera.com/blog/2014/11/how-to-do-near-real-time-sessionization-with-spark-streaming-and-apache-hadoop/

      A real customer using this in produce is blogged here
      http://blog.cloudera.com/blog/2015/03/how-edmunds-com-used-spark-streaming-to-build-a-near-real-time-dashboard/

      Please debate and let me know what I can do to make this happen.

      Attachments

        1. HBASE-13992.10.patch
          176 kB
          Theodore michael Malaska
        2. HBASE-13992.11.patch
          176 kB
          Theodore michael Malaska
        3. HBASE-13992.12.patch
          176 kB
          Theodore michael Malaska
        4. HBASE-13992.5.patch
          167 kB
          Theodore michael Malaska
        5. HBASE-13992.6.patch
          168 kB
          Theodore michael Malaska
        6. HBASE-13992.7.patch
          169 kB
          Theodore michael Malaska
        7. HBASE-13992.8.patch
          172 kB
          Theodore michael Malaska
        8. HBASE-13992.9.patch
          172 kB
          Theodore michael Malaska
        9. HBASE-13992.patch
          141 kB
          Theodore michael Malaska
        10. HBASE-13992.patch.3
          130 kB
          Theodore michael Malaska
        11. HBASE-13992.patch.4
          137 kB
          Theodore michael Malaska
        12. HBASE-13992.patch.5
          167 kB
          Theodore michael Malaska

        Issue Links

          duplicates

          New Feature - A new feature of the product, which has yet to be developed. HBASE-11482 Optimize HBase TableInput/OutputFormats for exposing tables and snapshots as Spark RDDs

          • Major - Major loss of function.
          • Closed
          is cloned by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-20570 CLONE - Integrate SparkOnHBase into HBase

          • Major - Major loss of function.
          • Closed
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-14149 Add Data Frame support for HBase-Spark Module

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-14150 Add BulkLoad functionality to HBase-Spark Module

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HBASE-14160 backport hbase-spark module to branch-1 and branch-2

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14158 Add documentation for Initial Release for HBase-Spark Module integration

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-14159 Resolve warning introduced by HBase-Spark module

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed
          is duplicated by

          New Feature - A new feature of the product, which has yet to be developed. HBASE-20570 CLONE - Integrate SparkOnHBase into HBase

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. SPARK-2447 Add common solution for sending upsert actions to HBase (put, deletes, and increment)

          • Major - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-11843 MapReduce classes shouldn't be in hbase-server

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          links to

          Web Link review board

          Activity

            People

              ted.m Theodore michael Malaska
              ted.m Theodore michael Malaska
              Votes:
              0 Vote for this issue
              Watchers:
              41 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: