Accumulo
  1. Accumulo
  2. ACCUMULO-638

Accumulo integration with Titan Graph database

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: contrib
    • Labels:
      None

      Description

      Titan graph database (https://github.com/thinkaurelius/titan) was recently open sourced (Apache 2 License). It is a "distributed graph database optimized for processing massive-scale graphs represented over a machine cluster". It currently supports storing data in both Cassandra and HBase. I believe Accumulo support should be added as well.

      Other references:

        Activity

        Hide
        Josh Elser added a comment -

        Looks like I can't directly comment on Github (maybe because I'm only looking at a diff and not an actually PR?)

        • Some commented stuff in the pom.xml. Should probably be deleted.
        • AccumuloKCVS.getSlice(KeySliceQuery, StoreTransaction) can probably just use a Scanner instead of a BatchScanner. Don't need to pay extra costs for a single row lookup.
        • Should AccumuloKCVS.getSlice(KeySliceQuery, StoreTransaction) throw a PermanentBackendException instead of Temporary for the IllegalArgumentException generated by a bad Range? If it fails once, it will continue to fail with the same data.
        • UTF-8 on the new Strings in ScanConfig.getIteratorSettings()?

        Otherwise it looks very nice. Saw correct looking usage of locality groups. Resources appeared to be getting closed.

        Show
        Josh Elser added a comment - Looks like I can't directly comment on Github (maybe because I'm only looking at a diff and not an actually PR?) Some commented stuff in the pom.xml. Should probably be deleted. AccumuloKCVS.getSlice(KeySliceQuery, StoreTransaction) can probably just use a Scanner instead of a BatchScanner. Don't need to pay extra costs for a single row lookup. Should AccumuloKCVS.getSlice(KeySliceQuery, StoreTransaction) throw a PermanentBackendException instead of Temporary for the IllegalArgumentException generated by a bad Range? If it fails once, it will continue to fail with the same data. UTF-8 on the new Strings in ScanConfig.getIteratorSettings()? Otherwise it looks very nice. Saw correct looking usage of locality groups. Resources appeared to be getting closed.
        Hide
        Russ Weeks added a comment -

        Yep, that would be awesome. Thanks, Josh!

        Show
        Russ Weeks added a comment - Yep, that would be awesome. Thanks, Josh!
        Hide
        Josh Elser added a comment -

        I'll try to make some time to give your changes a look. Would comments on Github be best?

        Show
        Josh Elser added a comment - I'll try to make some time to give your changes a look. Would comments on Github be best?
        Hide
        Russ Weeks added a comment -

        I think I'll pull these iterators into a separate commit; they're not actually required for the Titan integration but might be a nice optimization in some cases.

        Show
        Russ Weeks added a comment - I think I'll pull these iterators into a separate commit; they're not actually required for the Titan integration but might be a nice optimization in some cases.
        Hide
        Russ Weeks added a comment -

        There aren't any changes to Accumulo required here, it's strictly on the Titan side. The changes are here and they could definitely use a review.

        I'm a little concerned that there might not be any Accumulo experts that know Titan, nor any Titan experts that know Accumulo. I know a little bit about both projects but it's definitely possible that something's fallen through the cracks.

        Show
        Russ Weeks added a comment - There aren't any changes to Accumulo required here, it's strictly on the Titan side. The changes are here and they could definitely use a review. I'm a little concerned that there might not be any Accumulo experts that know Titan, nor any Titan experts that know Accumulo. I know a little bit about both projects but it's definitely possible that something's fallen through the cracks.
        Hide
        Russ Weeks added a comment -

        Thanks, Sean. Let's leave it as-is for now. I hope to have it done in the next couple weeks but it's tough to make a firm commitment.

        Show
        Russ Weeks added a comment - Thanks, Sean. Let's leave it as-is for now. I hope to have it done in the next couple weeks but it's tough to make a firm commitment.
        Hide
        Sean Busbey added a comment -

        If you can post your changes up as a patch (either on review board or as a github PR) then I'd be happy to help take a look.

        Show
        Sean Busbey added a comment - If you can post your changes up as a patch (either on review board or as a github PR) then I'd be happy to help take a look.
        Hide
        Sean Busbey added a comment -

        All yours Russ! Would you like me to set a target version? Or should we just go for whatever it lands on?

        Show
        Sean Busbey added a comment - All yours Russ! Would you like me to set a target version? Or should we just go for whatever it lands on?
        Hide
        Russ Weeks added a comment -

        Is it OK if I assign this to myself? Looks like there hasn't been much activity since it was created.

        I'd be very grateful for an extra pair of eyes on my work, if anybody has some spare cycles. I guess it's complicated by the fact that most of the code changes will be in the Titan codebase, but there are a few iterators that I think would be generally useful in Accumulo.

        Show
        Russ Weeks added a comment - Is it OK if I assign this to myself? Looks like there hasn't been much activity since it was created. I'd be very grateful for an extra pair of eyes on my work , if anybody has some spare cycles. I guess it's complicated by the fact that most of the code changes will be in the Titan codebase, but there are a few iterators that I think would be generally useful in Accumulo.
        Hide
        Mike Drob added a comment -

        Looks like somebody has already done this, although the result is not Apache Licensed so I won't link to it from here. It's a trivial web search to find it though.

        Show
        Mike Drob added a comment - Looks like somebody has already done this, although the result is not Apache Licensed so I won't link to it from here. It's a trivial web search to find it though.
        Hide
        Eric Newton added a comment -

        I pushed it to "None".. if Jason works on it, he can move it to 1.6.

        Show
        Eric Newton added a comment - I pushed it to "None".. if Jason works on it, he can move it to 1.6.
        Hide
        David Medinets added a comment -

        Can this ticket be pushed to v1.6?

        Show
        David Medinets added a comment - Can this ticket be pushed to v1.6?

          People

          • Assignee:
            Russ Weeks
            Reporter:
            Jason Trost
          • Votes:
            2 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:

              Development