Uploaded image for project: 'Apache Gora'
  1. Apache Gora
  2. GORA-167

Make Cassandra keyspace consistency configurable within gora.properties

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.2.1
    • 0.5
    • gora-cassandra
    • None

    Description

      Current in CassandraClient#checkKeyspace() consistency is hard coded such that consistency level is .ONE which permits consistency to wait until one replica has responded. This could be improved to enable users to specify other consistency profiles e.g.

      ANY: Wait until some replica has responded.
      ONE: Wait until one replica has responded.
      TWO: Wait until two replicas have responded.
      THREE: Wait until three replicas have responded.
      LOCAL_QUORUM: Wait for quorum on the datacenter the connection was stablished.
      EACH_QUORUM: Wait for quorum on each datacenter.
      QUORUM: Wait for a quorum of replicas (no matter which datacenter).
      ALL: Blocks for all the replicas before returning to the client.

      Configuration should be made available through gora.properties

      Attachments

        1. GORA-167v5.patch
          25 kB
          Renato Javier MarroquĂ­n Mogrovejo
        2. GORA-167v4.patch
          25 kB
          Renato Javier MarroquĂ­n Mogrovejo
        3. GORA-167v3.patch
          23 kB
          Lewis John McGibbney
        4. GORA-167v2.patch
          20 kB
          Lewis John McGibbney
        5. GORA-167.patch
          11 kB
          Lewis John McGibbney

        Issue Links

          Activity

            Patch for trunk.
            Can someone please check on the Properties param for checkKeyspace() method which now accepts the consistency level property from gora.properties.

            Is specifying this solely in gora.properties the best way to go, or should this be configurable programmatically as well?

            Thanks for any feedback.
            Lets put this one to bed.

            lewismc Lewis John McGibbney added a comment - Patch for trunk. Can someone please check on the Properties param for checkKeyspace() method which now accepts the consistency level property from gora.properties. Is specifying this solely in gora.properties the best way to go, or should this be configurable programmatically as well? Thanks for any feedback. Lets put this one to bed.

            Any comments on this one please? I am keen to see if Gora can recognize and implement (via Hectors configurable logic) consistency levels.

            lewismc Lewis John McGibbney added a comment - Any comments on this one please? I am keen to see if Gora can recognize and implement (via Hectors configurable logic) consistency levels.

            Hi Lewis,

            great This is in some way related to GORA-215, maybe I'll find time to do these things the same way, after this one is fixed.

            So, here my comments:

            • documentation of checkKeyspace() is a bit unclear to me:
            • "Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath: "?!
            • "Applications may add additional resources, which are loaded subsequent to these resources in the order they are added"
            • And you're missing a few "</p>"
            • I get test failures on my box:
              Test set: org.apache.gora.cassandra.store.TestCassandraStore
              -------------------------------------------------------------------------------
              Tests run: 28, Failures: 0, Errors: 13, Skipped: 0, Time elapsed: 65.264 sec <<< FAILURE!
              testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore)  Time elapsed: 2.729 sec  <<< ERROR!
              java.lang.NullPointerException: Null not allowed, number 1
                      at me.prettyprint.cassandra.utils.Assert.noneNull(Assert.java:25)
                      at me.prettyprint.cassandra.model.AbstractBasicQuery.<init>(AbstractBasicQuery.java:26)
                      at me.prettyprint.cassandra.model.AbstractQuery.<init>(AbstractQuery.java:15)
                      at me.prettyprint.cassandra.model.AbstractSliceQuery.<init>(AbstractSliceQuery.java:24)
                      at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.<init>(ThriftRangeSlicesQuery.java:41)
                      at me.prettyprint.hector.api.factory.HFactory.createRangeSlicesQuery(HFactory.java:447)
                      at org.apache.gora.cassandra.store.CassandraClient.execute(CassandraClient.java:374)
                      at org.apache.gora.cassandra.store.CassandraStore.addSubColumns(CassandraStore.java:158)
                      at org.apache.gora.cassandra.store.CassandraStore.execute(CassandraStore.java:146)
                      at org.apache.gora.query.impl.QueryBase.execute(QueryBase.java:71)
                      at org.apache.gora.store.DataStoreTestUtil.assertNumResults(DataStoreTestUtil.java:495)
                      at org.apache.gora.store.DataStoreTestUtil.assertEmptyResults(DataStoreTestUtil.java:490)
                      at org.apache.gora.store.DataStoreTestUtil.testTruncateSchema(DataStoreTestUtil.java:114)
                      at org.apache.gora.store.DataStoreTestBase.testTruncateSchema(DataStoreTestBase.java:152)
                      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                      at java.lang.reflect.Method.invoke(Method.java:597)
              [...]
              testPutNested(org.apache.gora.cassandra.store.TestCassandraStore)  Time elapsed: 2.239 sec  <<< ERROR!
              java.lang.NullPointerException
                      at org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:216)
                      at org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:352)
                      at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:228)
                      at org.apache.gora.store.DataStoreTestBase.testPutNested(DataStoreTestBase.java:195)
                      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                      at java.lang.reflect.Method.invoke(Method.java:597)
                      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
                      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
                      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
                      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
                      at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
                      at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
              [...]
              

              (I'm on trunk - rev1466780 - patch applied with a few offsets, plain trunk has no errors during testing)

            rherget Roland von Herget added a comment - Hi Lewis, great This is in some way related to GORA-215 , maybe I'll find time to do these things the same way, after this one is fixed. So, here my comments: documentation of checkKeyspace() is a bit unclear to me: "Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath: "?! "Applications may add additional resources, which are loaded subsequent to these resources in the order they are added" And you're missing a few "</p>" I get test failures on my box: Test set: org.apache.gora.cassandra.store.TestCassandraStore ------------------------------------------------------------------------------- Tests run: 28, Failures: 0, Errors: 13, Skipped: 0, Time elapsed: 65.264 sec <<< FAILURE! testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 2.729 sec <<< ERROR! java.lang.NullPointerException: Null not allowed, number 1 at me.prettyprint.cassandra.utils.Assert.noneNull(Assert.java:25) at me.prettyprint.cassandra.model.AbstractBasicQuery.<init>(AbstractBasicQuery.java:26) at me.prettyprint.cassandra.model.AbstractQuery.<init>(AbstractQuery.java:15) at me.prettyprint.cassandra.model.AbstractSliceQuery.<init>(AbstractSliceQuery.java:24) at me.prettyprint.cassandra.model.thrift.ThriftRangeSlicesQuery.<init>(ThriftRangeSlicesQuery.java:41) at me.prettyprint.hector.api.factory.HFactory.createRangeSlicesQuery(HFactory.java:447) at org.apache.gora.cassandra.store.CassandraClient.execute(CassandraClient.java:374) at org.apache.gora.cassandra.store.CassandraStore.addSubColumns(CassandraStore.java:158) at org.apache.gora.cassandra.store.CassandraStore.execute(CassandraStore.java:146) at org.apache.gora.query.impl.QueryBase.execute(QueryBase.java:71) at org.apache.gora.store.DataStoreTestUtil.assertNumResults(DataStoreTestUtil.java:495) at org.apache.gora.store.DataStoreTestUtil.assertEmptyResults(DataStoreTestUtil.java:490) at org.apache.gora.store.DataStoreTestUtil.testTruncateSchema(DataStoreTestUtil.java:114) at org.apache.gora.store.DataStoreTestBase.testTruncateSchema(DataStoreTestBase.java:152) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) [...] testPutNested(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 2.239 sec <<< ERROR! java.lang.NullPointerException at org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:216) at org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:352) at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:228) at org.apache.gora.store.DataStoreTestBase.testPutNested(DataStoreTestBase.java:195) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) [...] (I'm on trunk - rev1466780 - patch applied with a few offsets, plain trunk has no errors during testing)

            Hey Roland. Thank you for reviewing

            {bq} "Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath: "?! {bq}

            I actually have no idea where I got this one from! This is unlike me? Maybe I was hung over or something that day!

            {bq} "Applications may add additional resources, which are loaded subsequent to these resources in the order they are added" {bq}

            ditto!!!

            I can add the <p> syntax, this is not a problem. Thank you for your attention to detail.

            I wonder if you tried changing the keyspace consistency? Please correct me if I am wrong, but I think you mentioned previously that your applications which write Gora data to Cassandra are not living the in the same data centre and communicate across network? Are you able to change the consistency and describe your findings?

            lewismc Lewis John McGibbney added a comment - Hey Roland. Thank you for reviewing {bq} "Unless explicitly turned off, Hadoop by default specifies two resources, loaded in-order from the classpath: "?! {bq} I actually have no idea where I got this one from! This is unlike me? Maybe I was hung over or something that day! {bq} "Applications may add additional resources, which are loaded subsequent to these resources in the order they are added" {bq} ditto!!! I can add the <p> syntax, this is not a problem. Thank you for your attention to detail. I wonder if you tried changing the keyspace consistency? Please correct me if I am wrong, but I think you mentioned previously that your applications which write Gora data to Cassandra are not living the in the same data centre and communicate across network? Are you able to change the consistency and describe your findings?

            I can confirm that I also get loads of failures so I will cook another patch up.

            lewismc Lewis John McGibbney added a comment - I can confirm that I also get loads of failures so I will cook another patch up.

            Patch for trunk. The property from GORA.PROPERTIES is not being picked up,,, but I don't quite know why. Anyone that can look would make me happy

            lewismc Lewis John McGibbney added a comment - Patch for trunk. The property from GORA.PROPERTIES is not being picked up,,, but I don't quite know why. Anyone that can look would make me happy

            Patch for trunk.
            AFAICT this baby works with 8 nodes.
            All is OK
            AFAIK local tests pass.
            If you guys are able to review would be great. It is another addition to gora.properties though.

            lewismc Lewis John McGibbney added a comment - Patch for trunk. AFAICT this baby works with 8 nodes. All is OK AFAIK local tests pass. If you guys are able to review would be great. It is another addition to gora.properties though.

            Hi lewismc,

            This is a cool patch mate. This works as expected
            I think what we could do is to extend this property to also allow consistency levels per operation as Cassandra also allows. So maybe we should change the property name as the patch is currently using as a default "ColumnFamily" consistency level.
            So I think maybe using a different property name, something like: gora.cassandrastore.cf.consistency.level and gora.cassandrastore.read.consistency.level for when that gets in. Wdyt?
            And also IMHO the default consistency level should be the one provided by the client so users don't get more confused. Hector uses Quorum consistency level as a default.

            renato2099 Renato Javier MarroquĂ­n Mogrovejo added a comment - Hi lewismc , This is a cool patch mate. This works as expected I think what we could do is to extend this property to also allow consistency levels per operation as Cassandra also allows. So maybe we should change the property name as the patch is currently using as a default "ColumnFamily" consistency level. So I think maybe using a different property name, something like: gora.cassandrastore.cf.consistency.level and gora.cassandrastore.read.consistency.level for when that gets in. Wdyt? And also IMHO the default consistency level should be the one provided by the client so users don't get more confused. Hector uses Quorum consistency level as a default.

            ...also allow consistency levels per operation as Cassandra also allows.

            +1 definately

            So you're proposing the following as properties to be added to gora.properties

            gora.cassandrastore.cf.consistency.level (presumably this one accommodates writes into Cassandra?)
            gora.cassandrastore.read.consistency.level

            default consistency level should be the one provided by the client so users don't get more confused. Hector uses Quorum consistency level as a default.

            No probs at all. I agree here as well. This is my fault and it should be client preference by default.

            If you're keen to implement these changes please fire away, if not then say so and I will try to implement them.

            lewismc Lewis John McGibbney added a comment - ...also allow consistency levels per operation as Cassandra also allows. +1 definately So you're proposing the following as properties to be added to gora.properties gora.cassandrastore.cf.consistency.level (presumably this one accommodates writes into Cassandra?) gora.cassandrastore.read.consistency.level default consistency level should be the one provided by the client so users don't get more confused. Hector uses Quorum consistency level as a default. No probs at all. I agree here as well. This is my fault and it should be client preference by default. If you're keen to implement these changes please fire away, if not then say so and I will try to implement them.

            This is just an update to lewismc's patch.

            renato2099 Renato Javier MarroquĂ­n Mogrovejo added a comment - This is just an update to lewismc 's patch.

            So lewismc if you could review it, I could then commit it

            renato2099 Renato Javier MarroquĂ­n Mogrovejo added a comment - So lewismc if you could review it, I could then commit it

            Committed to trunk. Revision number r1575224.

            renato2099 Renato Javier MarroquĂ­n Mogrovejo added a comment - Committed to trunk. Revision number r1575224.

            Added some simple debugging.

            renato2099 Renato Javier MarroquĂ­n Mogrovejo added a comment - Added some simple debugging.
            hudson Hudson added a comment -

            SUCCESS: Integrated in gora-trunk #1025 (See https://builds.apache.org/job/gora-trunk/1025/)
            GORA-167 (rmarroquin: http://svn.apache.org/viewvc/gora/trunk/?view=rev&rev=1575224)

            • /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraClient.java
            • /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraStore.java
            • /gora/trunk/gora-cassandra/src/test/conf/gora.properties
            • /gora/trunk/gora-core/src/main/java/org/apache/gora/store/DataStoreFactory.java
            • /gora/trunk/gora-dynamodb/src/main/java/org/apache/gora/dynamodb/store/DynamoDBStore.java
            hudson Hudson added a comment - SUCCESS: Integrated in gora-trunk #1025 (See https://builds.apache.org/job/gora-trunk/1025/ ) GORA-167 (rmarroquin: http://svn.apache.org/viewvc/gora/trunk/?view=rev&rev=1575224 ) /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraClient.java /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraStore.java /gora/trunk/gora-cassandra/src/test/conf/gora.properties /gora/trunk/gora-core/src/main/java/org/apache/gora/store/DataStoreFactory.java /gora/trunk/gora-dynamodb/src/main/java/org/apache/gora/dynamodb/store/DynamoDBStore.java

            Are we we porting this to GORA_94 then we can close it out?

            lewismc Lewis John McGibbney added a comment - Are we we porting this to GORA_94 then we can close it out?

            This did not make it in to trunk a the backport of code from GORA-94 killed it off.
            We should port it to trunk.

            lewismc Lewis John McGibbney added a comment - This did not make it in to trunk a the backport of code from GORA-94 killed it off. We should port it to trunk.

            This issue was NOT ported over to master branch when we merged the GORA_94 changes into master.
            The two commits below merely make the commit again as per what was previously approved and committed.
            Thank you renato2099 for the patch, this is a nice one.
            All tests passing locally.

            commit 3cacf2a25f2f8937284d50805eeb353bf5a1eabc
            Author: Lewis John McGibbney <lewis.j.mcgibbney@jpl.nasa.gov>
            Date: Fri Aug 15 09:50:50 2014 -0700

            GORA-167 forward port of Make Cassandra keyspace consistency configurable within gora.properties

            commit 68302e213747700ca38cef546f05d0f15a3e99b4
            Author: Lewis John McGibbney <lewis.j.mcgibbney@jpl.nasa.gov>
            Date: Fri Aug 15 09:49:14 2014 -0700

            forward port of Make Cassandra keyspace consistency configurable within gora.properties

            lewismc Lewis John McGibbney added a comment - This issue was NOT ported over to master branch when we merged the GORA_94 changes into master. The two commits below merely make the commit again as per what was previously approved and committed. Thank you renato2099 for the patch, this is a nice one. All tests passing locally. commit 3cacf2a25f2f8937284d50805eeb353bf5a1eabc Author: Lewis John McGibbney <lewis.j.mcgibbney@jpl.nasa.gov> Date: Fri Aug 15 09:50:50 2014 -0700 GORA-167 forward port of Make Cassandra keyspace consistency configurable within gora.properties commit 68302e213747700ca38cef546f05d0f15a3e99b4 Author: Lewis John McGibbney <lewis.j.mcgibbney@jpl.nasa.gov> Date: Fri Aug 15 09:49:14 2014 -0700 forward port of Make Cassandra keyspace consistency configurable within gora.properties
            hudson Hudson added a comment -

            SUCCESS: Integrated in gora-trunk #1177 (See https://builds.apache.org/job/gora-trunk/1177/)
            GORA-167 forward port of Make Cassandra keyspace consistency configurable within gora.properties (lewis.j.mcgibbney: rev 3cacf2a25f2f8937284d50805eeb353bf5a1eabc)

            • CHANGES.txt
            hudson Hudson added a comment - SUCCESS: Integrated in gora-trunk #1177 (See https://builds.apache.org/job/gora-trunk/1177/ ) GORA-167 forward port of Make Cassandra keyspace consistency configurable within gora.properties (lewis.j.mcgibbney: rev 3cacf2a25f2f8937284d50805eeb353bf5a1eabc) CHANGES.txt

            People

              lewismc Lewis John McGibbney
              lewismc Lewis John McGibbney
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: