Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3340

Eventually Consistent Secondary Indexing via Coprocessors

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Coprocessors
    • Labels:
      None

      Description

      Secondary indexing support via coprocessors with an eventual consistency guarantee. Design to come.

        Issue Links

          Activity

          Hide
          apurtell Andrew Purtell added a comment -

          After reviewing the latest code drop from our internal team, we have decided it has architectural flaws significant enough to remove it from consideration at this time.

          Show
          apurtell Andrew Purtell added a comment - After reviewing the latest code drop from our internal team, we have decided it has architectural flaws significant enough to remove it from consideration at this time.
          Hide
          streamy Jonathan Gray added a comment -

          I'm not actively working on this but it's also a potential intern project at fb. A code drop on GitHub would be great and maybe we can work together. There are quite a few alternative directions to go for indexing. And an endless amount of development that could be done around APIs, schemas, filters, etc. So the more the merrier.

          The basic design I was thinking would be something similar to google percolator or what the Lily guys are doing (http://www.lilyproject.org/lily/about/playground/hbaserowlog/version/1)

          Show
          streamy Jonathan Gray added a comment - I'm not actively working on this but it's also a potential intern project at fb. A code drop on GitHub would be great and maybe we can work together. There are quite a few alternative directions to go for indexing. And an endless amount of development that could be done around APIs, schemas, filters, etc. So the more the merrier. The basic design I was thinking would be something similar to google percolator or what the Lily guys are doing ( http://www.lilyproject.org/lily/about/playground/hbaserowlog/version/1 )
          Hide
          tlipcon Todd Lipcon added a comment -

          great, thanks. if you have some kind of design doc for it that would be cool - we can see if it lines up with the kind of ideas we were thinking about

          Show
          tlipcon Todd Lipcon added a comment - great, thanks. if you have some kind of design doc for it that would be cool - we can see if it lines up with the kind of ideas we were thinking about
          Hide
          apurtell Andrew Purtell added a comment -

          Actively working on it. It's a starter project for them so we have had some back and forth about design issues. They are about to run a test with 1 billion rows and 3 indexes, however. Let me ask for a code drop and I'll post a link to the tree on this issue.

          Show
          apurtell Andrew Purtell added a comment - Actively working on it. It's a starter project for them so we have had some back and forth about design issues. They are about to run a test with 1 billion rows and 3 indexes, however. Let me ask for a code drop and I'll post a link to the tree on this issue.
          Hide
          tlipcon Todd Lipcon added a comment -

          github code drop would be cool - then we can help with it or see if we were planning on taking it in some different direction. Are they actively working on it or just got halfway through and moved on to other projects?

          Show
          tlipcon Todd Lipcon added a comment - github code drop would be cool - then we can help with it or see if we were planning on taking it in some different direction. Are they actively working on it or just got halfway through and moved on to other projects?
          Hide
          apurtell Andrew Purtell added a comment -

          We also have a team that has this I'd say half done. Could do a code drop on GitHub.

          Show
          apurtell Andrew Purtell added a comment - We also have a team that has this I'd say half done. Could do a code drop on GitHub.
          Hide
          tlipcon Todd Lipcon added a comment -

          Hey Jonathan, are you currently working on this? We have an intern who is interested.

          Show
          tlipcon Todd Lipcon added a comment - Hey Jonathan, are you currently working on this? We have an intern who is interested.
          Show
          ekoontz Eugene Koontz added a comment - See also http://wiki.apache.org/hadoop/Hbase/SecondaryIndexing
          Hide
          streamy Jonathan Gray added a comment -

          I'm not exactly sure because I'm not clear on how HBASE-2038 is going to be implemented. I think there are several different ideas floating around for indexing. This is one specific idea I'm thinking about working on.

          Show
          streamy Jonathan Gray added a comment - I'm not exactly sure because I'm not clear on how HBASE-2038 is going to be implemented. I think there are several different ideas floating around for indexing. This is one specific idea I'm thinking about working on.
          Hide
          hammer Jeff Hammerbacher added a comment -
          Show
          hammer Jeff Hammerbacher added a comment - How is this different from https://issues.apache.org/jira/browse/HBASE-2038?

            People

            • Assignee:
              Unassigned
              Reporter:
              streamy Jonathan Gray
            • Votes:
              4 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

              • Created:
                Updated:

                Development