Apache Gora
  1. Apache Gora
  2. GORA-114

Make hbase autoflush default to false and make autoflush configurable rather than hardcoded

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.2
    • Fix Version/s: 0.2
    • Component/s: gora-hbase
    • Labels:
      None

      Description

      In gora trunk, its hard coded so hbase will flush from the client every individual edit. This makes it slow trying to load a bunch of data in via gora.

        Activity

        Hide
        stack added a comment -

        A patch to read autoflush setting from Configuration. Also sets the default to be off so we batch up edits before flushing to the server.

        Show
        stack added a comment - A patch to read autoflush setting from Configuration. Also sets the default to be off so we batch up edits before flushing to the server.
        Hide
        Lewis John McGibbney added a comment -

        Hi Stack. Can you confirm if the bollean property would be included in hbase-site.xml? I am not so familiar with the configuration characteristics for HBase, but as far as I know this is the main configuration file. Thank you

        Show
        Lewis John McGibbney added a comment - Hi Stack. Can you confirm if the bollean property would be included in hbase-site.xml? I am not so familiar with the configuration characteristics for HBase, but as far as I know this is the main configuration file. Thank you
        Hide
        stack added a comment -

        Can you confirm if the bollean property would be included in hbase-site.xml?

        You'd include it only if you wanted to change the default.

        You are correct that hbase-site.xml is where all of the customization of a particular instance go.

        Anything else I can do to help this patch along Lewis? Any place you want me to put a bit of doc other than here in release note?

        The patch comes of my trying the nice gora derivative, https://github.com/keith-turner/goraci, at Keith's suggestion (In his README he suggests this is needed and I agree with him).

        Thanks.

        Show
        stack added a comment - Can you confirm if the bollean property would be included in hbase-site.xml? You'd include it only if you wanted to change the default. You are correct that hbase-site.xml is where all of the customization of a particular instance go. Anything else I can do to help this patch along Lewis? Any place you want me to put a bit of doc other than here in release note? The patch comes of my trying the nice gora derivative, https://github.com/keith-turner/goraci , at Keith's suggestion (In his README he suggests this is needed and I agree with him). Thanks.
        Hide
        Lewis John McGibbney added a comment -

        Absolutely I agree with this so thanks for opening the issue. It makes perfect sense to have this configurable. I'll add a short comment in the code to explain why the flush in configured so and where it can be configured from before I commit later today thank you very much.

        Show
        Lewis John McGibbney added a comment - Absolutely I agree with this so thanks for opening the issue. It makes perfect sense to have this configurable. I'll add a short comment in the code to explain why the flush in configured so and where it can be configured from before I commit later today thank you very much.
        Hide
        Lewis John McGibbney added a comment -

        Committed @ r1310996 in trunk. Thank you Stack. Please close this issue off when you have the time.

        Show
        Lewis John McGibbney added a comment - Committed @ r1310996 in trunk. Thank you Stack. Please close this issue off when you have the time.
        Hide
        Hudson added a comment -

        Integrated in gora-trunk #219 (See https://builds.apache.org/job/gora-trunk/219/)
        Commit to address GORA-114 and update to CHANGES.txt. Thank you Stack. (Revision 1310996)

        Result = SUCCESS
        lewismc :
        Files :

        • /gora/trunk/CHANGES.txt
        • /gora/trunk/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java
        Show
        Hudson added a comment - Integrated in gora-trunk #219 (See https://builds.apache.org/job/gora-trunk/219/ ) Commit to address GORA-114 and update to CHANGES.txt. Thank you Stack. (Revision 1310996) Result = SUCCESS lewismc : Files : /gora/trunk/CHANGES.txt /gora/trunk/gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java
        Hide
        Ferdy Galema added a comment - - edited

        Thanks for fixing this. This issue needs a small change to the HBaseTableConnection. Please see GORA-115 for this.

        Show
        Ferdy Galema added a comment - - edited Thanks for fixing this. This issue needs a small change to the HBaseTableConnection. Please see GORA-115 for this.
        Hide
        Hudson added a comment -

        Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/)
        Commit to address GORA-114 and update to CHANGES.txt. Thank you Stack. (Revision 7bf92af1f13d94257a43b8c48ec3c46b9f6614a0)

        Result = FAILURE
        lewismc :
        Files :

        • gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java
        • CHANGES.txt
        Show
        Hudson added a comment - Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/ ) Commit to address GORA-114 and update to CHANGES.txt. Thank you Stack. (Revision 7bf92af1f13d94257a43b8c48ec3c46b9f6614a0) Result = FAILURE lewismc : Files : gora-hbase/src/main/java/org/apache/gora/hbase/store/HBaseStore.java CHANGES.txt

          People

          • Assignee:
            Unassigned
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development