HBase
  1. HBase
  2. HBASE-3340

Eventually Consistent Secondary Indexing via Coprocessors

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Coprocessors
    • Labels:
      None

      Description

      Secondary indexing support via coprocessors with an eventual consistency guarantee. Design to come.

        Issue Links

          Activity

          Hide
          Andrew Purtell added a comment -

          After reviewing the latest code drop from our internal team, we have decided it has architectural flaws significant enough to remove it from consideration at this time.

          Show
          Andrew Purtell added a comment - After reviewing the latest code drop from our internal team, we have decided it has architectural flaws significant enough to remove it from consideration at this time.
          Hide
          Jonathan Gray added a comment -

          I'm not actively working on this but it's also a potential intern project at fb. A code drop on GitHub would be great and maybe we can work together. There are quite a few alternative directions to go for indexing. And an endless amount of development that could be done around APIs, schemas, filters, etc. So the more the merrier.

          The basic design I was thinking would be something similar to google percolator or what the Lily guys are doing (http://www.lilyproject.org/lily/about/playground/hbaserowlog/version/1)

          Show
          Jonathan Gray added a comment - I'm not actively working on this but it's also a potential intern project at fb. A code drop on GitHub would be great and maybe we can work together. There are quite a few alternative directions to go for indexing. And an endless amount of development that could be done around APIs, schemas, filters, etc. So the more the merrier. The basic design I was thinking would be something similar to google percolator or what the Lily guys are doing ( http://www.lilyproject.org/lily/about/playground/hbaserowlog/version/1 )
          Hide
          Todd Lipcon added a comment -

          great, thanks. if you have some kind of design doc for it that would be cool - we can see if it lines up with the kind of ideas we were thinking about

          Show
          Todd Lipcon added a comment - great, thanks. if you have some kind of design doc for it that would be cool - we can see if it lines up with the kind of ideas we were thinking about
          Hide
          Andrew Purtell added a comment -

          Actively working on it. It's a starter project for them so we have had some back and forth about design issues. They are about to run a test with 1 billion rows and 3 indexes, however. Let me ask for a code drop and I'll post a link to the tree on this issue.

          Show
          Andrew Purtell added a comment - Actively working on it. It's a starter project for them so we have had some back and forth about design issues. They are about to run a test with 1 billion rows and 3 indexes, however. Let me ask for a code drop and I'll post a link to the tree on this issue.
          Hide
          Todd Lipcon added a comment -

          github code drop would be cool - then we can help with it or see if we were planning on taking it in some different direction. Are they actively working on it or just got halfway through and moved on to other projects?

          Show
          Todd Lipcon added a comment - github code drop would be cool - then we can help with it or see if we were planning on taking it in some different direction. Are they actively working on it or just got halfway through and moved on to other projects?
          Hide
          Andrew Purtell added a comment -

          We also have a team that has this I'd say half done. Could do a code drop on GitHub.

          Show
          Andrew Purtell added a comment - We also have a team that has this I'd say half done. Could do a code drop on GitHub.
          Hide
          Todd Lipcon added a comment -

          Hey Jonathan, are you currently working on this? We have an intern who is interested.

          Show
          Todd Lipcon added a comment - Hey Jonathan, are you currently working on this? We have an intern who is interested.
          Show
          Eugene Koontz added a comment - See also http://wiki.apache.org/hadoop/Hbase/SecondaryIndexing
          Hide
          Jonathan Gray added a comment -

          I'm not exactly sure because I'm not clear on how HBASE-2038 is going to be implemented. I think there are several different ideas floating around for indexing. This is one specific idea I'm thinking about working on.

          Show
          Jonathan Gray added a comment - I'm not exactly sure because I'm not clear on how HBASE-2038 is going to be implemented. I think there are several different ideas floating around for indexing. This is one specific idea I'm thinking about working on.
          Hide
          Jeff Hammerbacher added a comment -
          Show
          Jeff Hammerbacher added a comment - How is this different from https://issues.apache.org/jira/browse/HBASE-2038?

            People

            • Assignee:
              Jacques Nadeau
              Reporter:
              Jonathan Gray
            • Votes:
              5 Vote for this issue
              Watchers:
              33 Start watching this issue

              Dates

              • Created:
                Updated:

                Development