Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.1.1-incubating
    • Fix Version/s: 0.2.1
    • Component/s: gora-cassandra
    • Labels:
      None

      Description

      As per this thread [1], it is absolutely essential that we get some tests for the Cassandra module. Therefore this task should act as a summary task to manage all test which are hereby proposed.

      Realistically, this is going to take a good while, so I have marked it for 0.2 and 0.3-incubating releases.

      [1] http://www.mail-archive.com/gora-dev@incubator.apache.org/msg00241.html

      1. GORA-53.patch
        45 kB
        Lewis John McGibbney
      2. GORA-53-final.patch
        45 kB
        Kazuomi Kashii
      3. GORA-53-v2.patch
        40 kB
        Lewis John McGibbney
      4. GORA-54-v8.patch
        36 kB
        Lewis John McGibbney
      5. GORA-test.patch
        53 kB
        Lewis John McGibbney
      6. test.txt
        1.22 MB
        Lewis John McGibbney

        Issue Links

          Activity

          Hide
          Lewis John McGibbney added a comment -

          Is it possible to initiate a conversation regarding what type of test to be running here and what aspects of the Gora-Cassandra configuration we wish to be testing?

          I'll have a more thorough look over the Cassandra module and comment myself in due course.

          Show
          Lewis John McGibbney added a comment - Is it possible to initiate a conversation regarding what type of test to be running here and what aspects of the Gora-Cassandra configuration we wish to be testing? I'll have a more thorough look over the Cassandra module and comment myself in due course.
          Hide
          Enis Soztutar added a comment -

          Hi Lewis,

          At first, we should target the default tests for DataStores. There is a DataStoreTestBase class which delegates tests to DataStoreTestUtil class in gora-core/src/test. The idea is to gather all data-store-agnostic tests under gora-core, and have each store module run all the tests against itself. So, similar to TestHBaseStore and TestSqlStore classes, we should have a TestCassandraStore test class which will run the generic tests. Note that the assertion of the data store operations would have to be written for cassandra.

          If we want to add generic tests that can be applicable to all data stores, we should add them in gora-core, and have cassandra module run those tests. We can also devise Cassandra-specific tests, which will go under gora-cassandra module.

          Show
          Enis Soztutar added a comment - Hi Lewis, At first, we should target the default tests for DataStores. There is a DataStoreTestBase class which delegates tests to DataStoreTestUtil class in gora-core/src/test. The idea is to gather all data-store-agnostic tests under gora-core, and have each store module run all the tests against itself. So, similar to TestHBaseStore and TestSqlStore classes, we should have a TestCassandraStore test class which will run the generic tests. Note that the assertion of the data store operations would have to be written for cassandra. If we want to add generic tests that can be applicable to all data stores, we should add them in gora-core, and have cassandra module run those tests. We can also devise Cassandra-specific tests, which will go under gora-cassandra module.
          Hide
          Lewis John McGibbney added a comment -

          Been looking at this Enis and understand the modularity now so plan to begin work on this shortly. Initially it will involve extending GoraTestDriver with GoraCassandraTestDriver then constructing TestCassandraStore. However while I'm at this could you suggest some other tests which would be useful to have for the Cassandra module. I've been in touch with some cassandra developers today so hopefully will hear of some boundary cases specific to Cassandra which would be nice to check for. Does anyone else have suggestions which I can get fired in to?

          Show
          Lewis John McGibbney added a comment - Been looking at this Enis and understand the modularity now so plan to begin work on this shortly. Initially it will involve extending GoraTestDriver with GoraCassandraTestDriver then constructing TestCassandraStore. However while I'm at this could you suggest some other tests which would be useful to have for the Cassandra module. I've been in touch with some cassandra developers today so hopefully will hear of some boundary cases specific to Cassandra which would be nice to check for. Does anyone else have suggestions which I can get fired in to?
          Hide
          Lewis John McGibbney added a comment -

          Hi Guys, this has been eating away at me so I thought I'd reach out for some help. I've started work on this and just wouldn't mind setting off in the right direction. I think the HBaseTestingUtility class [1] which we use in GoraHBaseTestDriver is great. It's really clean, I would imagine makes life easier to debug and obviously saves a power of time... so I'm trying to find a similar utility class which we can use with the GoraCassandraTestDriver. So far I've discovered the [2] EmbeddedServerHelper which would enable us to interact with Cassandra through the Hector Client. Adding the hector-test dependency to the maven build is a piece of cake but does this look OK to progress with or should the test dependencies be coming from the hadoop-test dependency... there is quite a lot of research to be done here so I was wondering if anyone is aware of this already???

          [1] https://svn.apache.org/repos/asf/hbase/branches/former_0.20/src/test/org/apache/hadoop/hbase/HBaseTestingUtility.java
          [2] https://github.com/rantav/hector/blob/master/test/src/main/java/me/prettyprint/hector/testutils/EmbeddedServerHelper.java

          Show
          Lewis John McGibbney added a comment - Hi Guys, this has been eating away at me so I thought I'd reach out for some help. I've started work on this and just wouldn't mind setting off in the right direction. I think the HBaseTestingUtility class [1] which we use in GoraHBaseTestDriver is great. It's really clean, I would imagine makes life easier to debug and obviously saves a power of time... so I'm trying to find a similar utility class which we can use with the GoraCassandraTestDriver. So far I've discovered the [2] EmbeddedServerHelper which would enable us to interact with Cassandra through the Hector Client. Adding the hector-test dependency to the maven build is a piece of cake but does this look OK to progress with or should the test dependencies be coming from the hadoop-test dependency... there is quite a lot of research to be done here so I was wondering if anyone is aware of this already??? [1] https://svn.apache.org/repos/asf/hbase/branches/former_0.20/src/test/org/apache/hadoop/hbase/HBaseTestingUtility.java [2] https://github.com/rantav/hector/blob/master/test/src/main/java/me/prettyprint/hector/testutils/EmbeddedServerHelper.java
          Hide
          Lewis John McGibbney added a comment -

          first scrape at addressing this issue. This is nowhere near suitable but I am attaching it this early on so I don't loose it or revert it when mucking around.

          Show
          Lewis John McGibbney added a comment - first scrape at addressing this issue. This is nowhere near suitable but I am attaching it this early on so I don't loose it or revert it when mucking around.
          Hide
          Lewis John McGibbney added a comment -

          a bit more progress on this one, but still no where near ready.

          Show
          Lewis John McGibbney added a comment - a bit more progress on this one, but still no where near ready.
          Hide
          Lewis John McGibbney added a comment -

          The attached patch is 'nearly' there. There are obvious problems which need ironed out but I thought best to attach what I have.

          Show
          Lewis John McGibbney added a comment - The attached patch is 'nearly' there. There are obvious problems which need ironed out but I thought best to attach what I have.
          Hide
          Lewis John McGibbney added a comment -

          most up to date patch. Tests still failing and I can't seem to crack why. I've added a check for the cassandrastart, preventing the server from shutting down and reinitializing at each test in the suite.

          Show
          Lewis John McGibbney added a comment - most up to date patch. Tests still failing and I can't seem to crack why. I've added a check for the cassandrastart, preventing the server from shutting down and reinitializing at each test in the suite.
          Hide
          Lewis John McGibbney added a comment -

          Still not quite there

          Show
          Lewis John McGibbney added a comment - Still not quite there
          Hide
          Lewis John McGibbney added a comment -

          One last push!!!

          Show
          Lewis John McGibbney added a comment - One last push!!!
          Hide
          Lewis John McGibbney added a comment -

          Most up-to-date patch on this issue. Still work to be done though.

          Show
          Lewis John McGibbney added a comment - Most up-to-date patch on this issue. Still work to be done though.
          Hide
          Lewis John McGibbney added a comment -

          This patch has a problem as we removed cassandra-all from the parent pom therefore the GoraCassandraTestDriver needs to be rewritten to use hectors embeddedserverhelper plus this dependency needs to be written into the parent and gora-cassandra/pom.xml's
          This is not really a headache as the class is very well written an very easy to use but I wanted to get opinions on what is best to do.

          Show
          Lewis John McGibbney added a comment - This patch has a problem as we removed cassandra-all from the parent pom therefore the GoraCassandraTestDriver needs to be rewritten to use hectors embeddedserverhelper plus this dependency needs to be written into the parent and gora-cassandra/pom.xml's This is not really a headache as the class is very well written an very easy to use but I wanted to get opinions on what is best to do.
          Hide
          Kazuomi Kashii added a comment -

          org.apache.cassandra.thrift.CassandraDaemon is not included in apache-cassandra-thrift-1.x.y.jar,
          but in apache-cassandra-1.x.y.jar,
          so dependency to cassandra-all should be required for GoraCassandraTestDriver.

          Even if GoraCassandraTestDriver is rewritten with Hector's EmbeddedServerHelper,
          it still requires CassandraDaemon at runtime.

          As long as using CassandraDaemon in test, I think it is the only way to include cassandra-all in dependency.

          Show
          Kazuomi Kashii added a comment - org.apache.cassandra.thrift.CassandraDaemon is not included in apache-cassandra-thrift-1.x.y.jar, but in apache-cassandra-1.x.y.jar, so dependency to cassandra-all should be required for GoraCassandraTestDriver. Even if GoraCassandraTestDriver is rewritten with Hector's EmbeddedServerHelper, it still requires CassandraDaemon at runtime. As long as using CassandraDaemon in test, I think it is the only way to include cassandra-all in dependency.
          Hide
          Kazuomi Kashii added a comment -

          If we can add dependency of cassandra-all only to "test" target, it should be better,
          but I am not so familiar with Maven's pom.xml, so I am not sure if it is possible.

          Show
          Kazuomi Kashii added a comment - If we can add dependency of cassandra-all only to "test" target, it should be better, but I am not so familiar with Maven's pom.xml, so I am not sure if it is possible.
          Hide
          Kazuomi Kashii added a comment -

          gora-cassandra/src/test/conf/cassandra.yaml has:
          sliced_buffer_size_in_kb: 64
          but Cassandra 1.1.x does not accept this property.

          After removing it, the test itself can run (but with failures).

          Show
          Kazuomi Kashii added a comment - gora-cassandra/src/test/conf/cassandra.yaml has: sliced_buffer_size_in_kb: 64 but Cassandra 1.1.x does not accept this property. After removing it, the test itself can run (but with failures).
          Hide
          Lewis John McGibbney added a comment -

          I get 1 failure and 52 errors. A number of complains in all honesty.
          I'm firstly going to investigate the following one as it occurs on the first test in the suite and also consistently throughout the entire test suite.

          testNewInstance(org.apache.gora.cassandra.store.TestCassandraStore)  Time elapsed: 19.749 sec  <<< ERROR!
          me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.)
          	at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:45)
          	at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:231)
          	at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:218)
          	at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103)
          	at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246)
          	at me.prettyprint.cassandra.service.AbstractCluster.dropKeyspace(AbstractCluster.java:235)
          	at me.prettyprint.cassandra.service.AbstractCluster.dropKeyspace(AbstractCluster.java:213)
          	at org.apache.gora.cassandra.store.CassandraClient.dropKeyspace(CassandraClient.java:128)
          	at org.apache.gora.cassandra.store.CassandraStore.deleteSchema(CassandraStore.java:118)
          	at org.apache.gora.GoraTestDriver.tearDown(GoraTestDriver.java:92)
          	at org.apache.gora.store.DataStoreTestBase.tearDown(DataStoreTestBase.java:112)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
          	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
          	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36)
          	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
          	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
          	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
          	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
          	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
          	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
          	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
          	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
          	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
          	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
          	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
          	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
          	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          	at java.lang.reflect.Method.invoke(Method.java:597)
          	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
          	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
          	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
          	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
          	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
          Caused by: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.)
          	at org.apache.cassandra.thrift.Cassandra$system_drop_keyspace_result.read(Cassandra.java:30808)
          	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
          	at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_keyspace(Cassandra.java:1314)
          	at org.apache.cassandra.thrift.Cassandra$Client.system_drop_keyspace(Cassandra.java:1301)
          	at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:225)
          	... 40 more
          
          Show
          Lewis John McGibbney added a comment - I get 1 failure and 52 errors. A number of complains in all honesty. I'm firstly going to investigate the following one as it occurs on the first test in the suite and also consistently throughout the entire test suite. testNewInstance(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 19.749 sec <<< ERROR! me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.) at me.prettyprint.cassandra.service.ExceptionsTranslatorImpl.translate(ExceptionsTranslatorImpl.java:45) at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:231) at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:218) at me.prettyprint.cassandra.service.Operation.executeAndSetResult(Operation.java:103) at me.prettyprint.cassandra.connection.HConnectionManager.operateWithFailover(HConnectionManager.java:246) at me.prettyprint.cassandra.service.AbstractCluster.dropKeyspace(AbstractCluster.java:235) at me.prettyprint.cassandra.service.AbstractCluster.dropKeyspace(AbstractCluster.java:213) at org.apache.gora.cassandra.store.CassandraClient.dropKeyspace(CassandraClient.java:128) at org.apache.gora.cassandra.store.CassandraStore.deleteSchema(CassandraStore.java:118) at org.apache.gora.GoraTestDriver.tearDown(GoraTestDriver.java:92) at org.apache.gora.store.DataStoreTestBase.tearDown(DataStoreTestBase.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:36) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74) Caused by: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.) at org.apache.cassandra.thrift.Cassandra$system_drop_keyspace_result.read(Cassandra.java:30808) at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78) at org.apache.cassandra.thrift.Cassandra$Client.recv_system_drop_keyspace(Cassandra.java:1314) at org.apache.cassandra.thrift.Cassandra$Client.system_drop_keyspace(Cassandra.java:1301) at me.prettyprint.cassandra.service.AbstractCluster$5.execute(AbstractCluster.java:225) ... 40 more
          Hide
          Kazuomi Kashii added a comment - - edited

          It is because of InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.)
          at org.apache.gora.cassandra.store.CassandraClient.dropKeyspace(CassandraClient.java:128)

          try

          { this.cluster.dropKeyspace(this.cassandraMapping.getKeyspaceName()); }

          catch (Exception e)

          { LOG.info(e.toString()); }

          would fix them (we may want better exception handling..), but we see another ones.

          Tests run: 28, Failures: 0, Errors: 27, Skipped: 0, Time elapsed: 2.076 sec <<< FAILURE!
          testCreateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.002 sec <<< ERROR!
          java.lang.NullPointerException
          at org.apache.gora.store.DataStoreTestUtil.testCreateEmployeeSchema(DataStoreTestUtil.java:102)
          at org.apache.gora.store.DataStoreTestBase.testCreateSchema(DataStoreTestBase.java:129)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          ...

          It seems that dataset has not been set (or not passed) at the following method.

          public static void testCreateEmployeeSchema(DataStore<String, Employee> dataStore)
          throws IOException

          { dataStore.createSchema(); //should not throw exception dataStore.createSchema(); }

          It is called by DataStoreTestBase:

          @Test
          public void testCreateSchema() throws Exception

          { log.info("test method: testCreateSchema"); DataStoreTestUtil.testCreateEmployeeSchema(employeeStore); assertSchemaExists("Employee"); }

          so employeeStore has not been set before that.

          Show
          Kazuomi Kashii added a comment - - edited It is because of InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.) at org.apache.gora.cassandra.store.CassandraClient.dropKeyspace(CassandraClient.java:128) try { this.cluster.dropKeyspace(this.cassandraMapping.getKeyspaceName()); } catch (Exception e) { LOG.info(e.toString()); } would fix them (we may want better exception handling..), but we see another ones. Tests run: 28, Failures: 0, Errors: 27, Skipped: 0, Time elapsed: 2.076 sec <<< FAILURE! testCreateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.002 sec <<< ERROR! java.lang.NullPointerException at org.apache.gora.store.DataStoreTestUtil.testCreateEmployeeSchema(DataStoreTestUtil.java:102) at org.apache.gora.store.DataStoreTestBase.testCreateSchema(DataStoreTestBase.java:129) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ... It seems that dataset has not been set (or not passed) at the following method. public static void testCreateEmployeeSchema(DataStore<String, Employee> dataStore) throws IOException { dataStore.createSchema(); //should not throw exception dataStore.createSchema(); } It is called by DataStoreTestBase: @Test public void testCreateSchema() throws Exception { log.info("test method: testCreateSchema"); DataStoreTestUtil.testCreateEmployeeSchema(employeeStore); assertSchemaExists("Employee"); } so employeeStore has not been set before that.
          Hide
          Kazuomi Kashii added a comment -

          It seems that setUp() is called at the beginning (twice), but not called every time,
          though tearDown() is called every time.

          ...
          12/07/12 09:04:58 INFO store.CassandraMapping: Added super column family: 'sc'
          12/07/12 09:04:58 INFO gora.GoraTestDriver: setting up test
          12/07/12 09:04:58 INFO store.CassandraClient: Keyspace 'Employee' in cluster 'Gora Cassandra Test Cluster' was created on host 'localhost'
          12/07/12 09:04:58 INFO store.CassandraClient: Keyspace 'Employee' in cluster 'Gora Cassandra Test Cluster' was created on host 'localhost'
          12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testNewInstance
          12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test
          12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test
          12/07/12 09:04:58 INFO store.CassandraClient: me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.)
          12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testCreateSchema
          12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test
          12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test
          12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testAutoCreateSchema
          12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test
          12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test
          12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testTruncateSchema
          12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test
          12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test
          12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testDeleteSchema
          12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test
          12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test
          ...

          Show
          Kazuomi Kashii added a comment - It seems that setUp() is called at the beginning (twice), but not called every time, though tearDown() is called every time. ... 12/07/12 09:04:58 INFO store.CassandraMapping: Added super column family: 'sc' 12/07/12 09:04:58 INFO gora.GoraTestDriver: setting up test 12/07/12 09:04:58 INFO store.CassandraClient: Keyspace 'Employee' in cluster 'Gora Cassandra Test Cluster' was created on host 'localhost' 12/07/12 09:04:58 INFO store.CassandraClient: Keyspace 'Employee' in cluster 'Gora Cassandra Test Cluster' was created on host 'localhost' 12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testNewInstance 12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test 12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test 12/07/12 09:04:58 INFO store.CassandraClient: me.prettyprint.hector.api.exceptions.HInvalidRequestException: InvalidRequestException(why:Cannot drop non existing keyspace 'Employee'.) 12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testCreateSchema 12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test 12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test 12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testAutoCreateSchema 12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test 12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test 12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testTruncateSchema 12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test 12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test 12/07/12 09:04:58 INFO store.DataStoreTestBase: test method: testDeleteSchema 12/07/12 09:04:58 INFO store.DataStoreTestBase: tearing down test 12/07/12 09:04:58 INFO gora.GoraTestDriver: tearing down test ...
          Hide
          Kazuomi Kashii added a comment - - edited

          I have updated setUp() method of DataStoreTestBase.java as follows:

          public void setUp() throws Exception {
          //There is an issue in JUnit 4 tests in Eclipse where TestSqlStore static
          //methods are not called BEFORE setUpClass. I think this is a bug in
          //JUnitRunner in Eclipse. Below is a workaround for that problem.
          if(!setUpClassCalled)

          { setUpClass(); }

          // <========================= I moved this bracket from the end.

          log.info("setting up test");
          if(testDriver != null)

          { employeeStore = testDriver.createDataStore(String.class, Employee.class); webPageStore = testDriver.createDataStore(String.class, WebPage.class); testDriver.setUp(); }

          else

          { employeeStore = createEmployeeDataStore(); webPageStore = createWebPageDataStore(); employeeStore.truncateSchema(); webPageStore.truncateSchema(); }

          // } <==== originally here !!
          }

          then, the test itself began to work, but I got the following numbers:

          Tests run: 51, Failures: 1, Errors: 48, Skipped: 0

          Show
          Kazuomi Kashii added a comment - - edited I have updated setUp() method of DataStoreTestBase.java as follows: public void setUp() throws Exception { //There is an issue in JUnit 4 tests in Eclipse where TestSqlStore static //methods are not called BEFORE setUpClass. I think this is a bug in //JUnitRunner in Eclipse. Below is a workaround for that problem. if(!setUpClassCalled) { setUpClass(); } // <========================= I moved this bracket from the end. log.info("setting up test"); if(testDriver != null) { employeeStore = testDriver.createDataStore(String.class, Employee.class); webPageStore = testDriver.createDataStore(String.class, WebPage.class); testDriver.setUp(); } else { employeeStore = createEmployeeDataStore(); webPageStore = createWebPageDataStore(); employeeStore.truncateSchema(); webPageStore.truncateSchema(); } // } <==== originally here !! } then, the test itself began to work, but I got the following numbers: Tests run: 51, Failures: 1, Errors: 48, Skipped: 0
          Hide
          Lewis John McGibbney added a comment -

          I'll look at this tonight as well Kaz. Great input.

          Show
          Lewis John McGibbney added a comment - I'll look at this tonight as well Kaz. Great input.
          Hide
          Kazuomi Kashii added a comment -

          Most of errors are "name is null" during tearDown() as follows:

          Tests run: 51, Failures: 1, Errors: 48, Skipped: 0, Time elapsed: 92.342 sec <<< FAILURE!
          testAutoCreateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.31 sec <<< ERROR!
          java.lang.IllegalArgumentException: name is null
          at me.prettyprint.cassandra.utils.Assert.notNull(Assert.java:18)
          at me.prettyprint.cassandra.model.HColumnImpl.<init>(HColumnImpl.java:36)
          at me.prettyprint.hector.api.factory.HFactory.createColumn(HFactory.java:608)
          at org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:152)
          at org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:336)
          at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:227)
          at org.apache.gora.cassandra.store.CassandraStore.close(CassandraStore.java:94)
          at org.apache.gora.GoraTestDriver.tearDown(GoraTestDriver.java:93)
          at org.apache.gora.store.DataStoreTestBase.tearDown(DataStoreTestBase.java:115)
          ...

          Show
          Kazuomi Kashii added a comment - Most of errors are "name is null" during tearDown() as follows: Tests run: 51, Failures: 1, Errors: 48, Skipped: 0, Time elapsed: 92.342 sec <<< FAILURE! testAutoCreateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.31 sec <<< ERROR! java.lang.IllegalArgumentException: name is null at me.prettyprint.cassandra.utils.Assert.notNull(Assert.java:18) at me.prettyprint.cassandra.model.HColumnImpl.<init>(HColumnImpl.java:36) at me.prettyprint.hector.api.factory.HFactory.createColumn(HFactory.java:608) at org.apache.gora.cassandra.store.CassandraClient.addColumn(CassandraClient.java:152) at org.apache.gora.cassandra.store.CassandraStore.addOrUpdateField(CassandraStore.java:336) at org.apache.gora.cassandra.store.CassandraStore.flush(CassandraStore.java:227) at org.apache.gora.cassandra.store.CassandraStore.close(CassandraStore.java:94) at org.apache.gora.GoraTestDriver.tearDown(GoraTestDriver.java:93) at org.apache.gora.store.DataStoreTestBase.tearDown(DataStoreTestBase.java:115) ...
          Hide
          Kazuomi Kashii added a comment -

          At CassandraClient.java:

          public void addColumn(K key, String fieldName, Object value) {
          if (value == null)

          { return; }

          ByteBuffer byteBuffer = toByteBuffer(value);

          String columnFamily = this.cassandraMapping.getFamily(fieldName);
          String columnName = this.cassandraMapping.getColumn(fieldName);

          LOG.info("columnFamily=" + columnFamily + " columnName=" + columnName + " value=" + value); // for debug

          this.mutator.insert(key, columnFamily, HFactory.createColumn(columnName, byteBuffer, StringSerializer.get(), ByteBufferSerializer.get()));
          }

          It printed:
          12/07/12 10:40:46 INFO store.CassandraClient: columnFamily=p columnName=null value=Random Joe

          Show
          Kazuomi Kashii added a comment - At CassandraClient.java: public void addColumn(K key, String fieldName, Object value) { if (value == null) { return; } ByteBuffer byteBuffer = toByteBuffer(value); String columnFamily = this.cassandraMapping.getFamily(fieldName); String columnName = this.cassandraMapping.getColumn(fieldName); LOG.info("columnFamily=" + columnFamily + " columnName=" + columnName + " value=" + value); // for debug this.mutator.insert(key, columnFamily, HFactory.createColumn(columnName, byteBuffer, StringSerializer.get(), ByteBufferSerializer.get())); } It printed: 12/07/12 10:40:46 INFO store.CassandraClient: columnFamily=p columnName=null value=Random Joe
          Hide
          Kazuomi Kashii added a comment -

          I updated gora-cassandra/src/test/conf/gora-cassandra-mapping.xml to use "qualifier" instead of "path" like:

          <class name="org.apache.gora.examples.generated.Employee" keyClass="java.lang.String" keyspace="Employee">
          <field name="name" family="p" qualifier="info:nm"/>
          <field name="dateOfBirth" family="p" qualifier="info:db"/>
          <field name="ssn" family="p" qualifier="info:sn"/>
          <field name="salary" family="p" qualifier="info:sl"/>
          </class>

          then, it prints:
          12/07/12 10:50:30 INFO store.CassandraClient: columnFamily=p columnName=info:nm value=Random Joe

          However, it seems that Avro version is conflicted between
          avro-1.3.3.jar and avro-1.4.0-cassandra-1.jar.
          The later one was added because of cassandra-all.

          testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.328 sec <<< ERROR!
          java.lang.NoSuchMethodError: org.apache.avro.generic.GenericArray.add(Ljava/lang/Object;)V
          at org.apache.gora.examples.generated.WebPage.addToParsedContent(WebPage.java:101)
          at org.apache.gora.examples.WebPageDataCreator.createWebPageData(WebPageDataCreator.java:120)
          at org.apache.gora.store.DataStoreTestUtil.testTruncateSchema(DataStoreTestUtil.java:111)
          at org.apache.gora.store.DataStoreTestBase.testTruncateSchema(DataStoreTestBase.java:152)

          Show
          Kazuomi Kashii added a comment - I updated gora-cassandra/src/test/conf/gora-cassandra-mapping.xml to use "qualifier" instead of "path" like: <class name="org.apache.gora.examples.generated.Employee" keyClass="java.lang.String" keyspace="Employee"> <field name="name" family="p" qualifier="info:nm"/> <field name="dateOfBirth" family="p" qualifier="info:db"/> <field name="ssn" family="p" qualifier="info:sn"/> <field name="salary" family="p" qualifier="info:sl"/> </class> then, it prints: 12/07/12 10:50:30 INFO store.CassandraClient: columnFamily=p columnName=info:nm value=Random Joe However, it seems that Avro version is conflicted between avro-1.3.3.jar and avro-1.4.0-cassandra-1.jar. The later one was added because of cassandra-all. testTruncateSchema(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.328 sec <<< ERROR! java.lang.NoSuchMethodError: org.apache.avro.generic.GenericArray.add(Ljava/lang/Object;)V at org.apache.gora.examples.generated.WebPage.addToParsedContent(WebPage.java:101) at org.apache.gora.examples.WebPageDataCreator.createWebPageData(WebPageDataCreator.java:120) at org.apache.gora.store.DataStoreTestUtil.testTruncateSchema(DataStoreTestUtil.java:111) at org.apache.gora.store.DataStoreTestBase.testTruncateSchema(DataStoreTestBase.java:152)
          Hide
          Kazuomi Kashii added a comment -

          We need cassandra-all for org.apache.cassandra.thrift.CassandraDaemon,
          and cassadra-all depends on avro-1.4.0-cassandra-1.jar,
          but we don't need avro-1.4.0-cassandra-1.jar at all,
          and it conflicts with avro-1.3.3.jar.
          We need to solve this dependency issue.

          Show
          Kazuomi Kashii added a comment - We need cassandra-all for org.apache.cassandra.thrift.CassandraDaemon, and cassadra-all depends on avro-1.4.0-cassandra-1.jar, but we don't need avro-1.4.0-cassandra-1.jar at all, and it conflicts with avro-1.3.3.jar. We need to solve this dependency issue.
          Hide
          Kazuomi Kashii added a comment -

          I manually edited ~/.m2/repository/org/apache/cassandra/cassandra-all/1.1.0/cassandra-all-1.1.0.pom
          to delete the followings from two places:

          <dependency>
          <groupId>org.apache.cassandra.deps</groupId>
          <artifactId>avro</artifactId>
          <version>1.4.0-cassandra-1</version>
          <exclusions>
          <exclusion>
          <artifactId>netty</artifactId>
          <groupId>org.jboss.netty</groupId>
          </exclusion>
          <exclusion>
          <artifactId>paranamer</artifactId>
          <groupId>com.thoughtworks.paranamer</groupId>
          </exclusion>
          <exclusion>
          <artifactId>paranamer-ant</artifactId>
          <groupId>com.thoughtworks.paranamer</groupId>
          </exclusion>
          <exclusion>
          <artifactId>velocity</artifactId>
          <groupId>org.apache.velocity</groupId>
          </exclusion>
          </exclusions>
          </dependency>

          and it temporarily fixed the dependency issue.

          However, I found the fatal problem in the code on CassandraMapping.java which supports only one keyspace and class,
          so the test succeeds for Employee but fails for WebPage.
          This should be filed as a separate bug.

          Show
          Kazuomi Kashii added a comment - I manually edited ~/.m2/repository/org/apache/cassandra/cassandra-all/1.1.0/cassandra-all-1.1.0.pom to delete the followings from two places: <dependency> <groupId>org.apache.cassandra.deps</groupId> <artifactId>avro</artifactId> <version>1.4.0-cassandra-1</version> <exclusions> <exclusion> <artifactId>netty</artifactId> <groupId>org.jboss.netty</groupId> </exclusion> <exclusion> <artifactId>paranamer</artifactId> <groupId>com.thoughtworks.paranamer</groupId> </exclusion> <exclusion> <artifactId>paranamer-ant</artifactId> <groupId>com.thoughtworks.paranamer</groupId> </exclusion> <exclusion> <artifactId>velocity</artifactId> <groupId>org.apache.velocity</groupId> </exclusion> </exclusions> </dependency> and it temporarily fixed the dependency issue. However, I found the fatal problem in the code on CassandraMapping.java which supports only one keyspace and class, so the test succeeds for Employee but fails for WebPage. This should be filed as a separate bug.
          Hide
          Kazuomi Kashii added a comment -

          I have created the patch for GORA-148, but found another large issue in the test.

          gora-core/src/examples/avro/webpage.json is:

          {
          "type": "record",
          "name": "WebPage",
          "namespace": "org.apache.gora.examples.generated",
          "fields" : [

          {"name": "url", "type": "string"}

          ,

          {"name": "content", "type": "bytes"}

          ,
          {"name": "parsedContent", "type": {"type":"array", "items": "string"}},
          {"name": "outlinks", "type": {"type":"map", "values":"string"}},
          {"name": "metadata", "type": {
          "name": "Metadata",
          "type": "record",
          "namespace": "org.apache.gora.examples.generated",
          "fields": [

          {"name": "version", "type": "int"}

          ,
          {"name": "data", "type": {"type": "map", "values": "string"}}
          ]
          }}
          ]
          }

          The "metadata" field is RECORD type, and it has "data" field which is MAP type inside.
          Currently, such nested complex type is not supported in gora-cassandra, even with super column family.

          Even the latest GORA-142-v3.patch does not support such nested complex type,
          but it is not so difficult to add a new serializer for this purpose.

          How do you think ?
          In order to pass the test, we need either
          1) to remove nested complex type from webpage.json; or
          2) to implement a new serializer with GORA-142.

          Show
          Kazuomi Kashii added a comment - I have created the patch for GORA-148 , but found another large issue in the test. gora-core/src/examples/avro/webpage.json is: { "type": "record", "name": "WebPage", "namespace": "org.apache.gora.examples.generated", "fields" : [ {"name": "url", "type": "string"} , {"name": "content", "type": "bytes"} , {"name": "parsedContent", "type": {"type":"array", "items": "string"}}, {"name": "outlinks", "type": {"type":"map", "values":"string"}}, {"name": "metadata", "type": { "name": "Metadata", "type": "record", "namespace": "org.apache.gora.examples.generated", "fields": [ {"name": "version", "type": "int"} , {"name": "data", "type": {"type": "map", "values": "string"}} ] }} ] } The "metadata" field is RECORD type, and it has "data" field which is MAP type inside. Currently, such nested complex type is not supported in gora-cassandra, even with super column family. Even the latest GORA-142 -v3.patch does not support such nested complex type, but it is not so difficult to add a new serializer for this purpose. How do you think ? In order to pass the test, we need either 1) to remove nested complex type from webpage.json; or 2) to implement a new serializer with GORA-142 .
          Hide
          Lewis John McGibbney added a comment -

          Hi Kaz. This is a combination of the two patches for GORA-53 and GORA-148 to try getting the tests working. After applying this I get a hanging cassandra server as it mentions there are too many files open. Typical output then the eventual stack traces can be seen in the accompanying test.txt

          Show
          Lewis John McGibbney added a comment - Hi Kaz. This is a combination of the two patches for GORA-53 and GORA-148 to try getting the tests working. After applying this I get a hanging cassandra server as it mentions there are too many files open. Typical output then the eventual stack traces can be seen in the accompanying test.txt
          Hide
          Kazuomi Kashii added a comment -

          Finally, gora-cassandra tests started working after some additional changes.
          1) Applied GORA-148, GORA-142, and GORA-149 patches.
          2) Uses cassandra-all 1.1.0 (not 1.1.1)
          3) gora-cassandra/src/test/conf/gora-cassandra-mapping.xml
          keyspace: WebPage -> webpage
          family="sc" for parsedContent, outlinks, and metadata

          Tests run: 28, Failures: 9, Errors: 5, Skipped: 0

          Results :

          Failed tests: testSchemaExists(org.apache.gora.cassandra.store.TestCassandraStore)
          testUpdate(org.apache.gora.cassandra.store.TestCassandraStore): expected:<[content0]> but was:<[?A@B@@@Pget_range_slices@@@5^O^@^@L@@@^A^K^@A@@@Nhttp://a.com/a^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Nhttp://a.com/a(..)
          testGetWebPage(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@=O@^@L@@@^A^K^@A@@@Ohttp://foo.com/^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Ohttp://foo.com/(..)
          testQuery(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@O^O^@^@L@@@(..)
          testQueryStartKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@Q^O^@^@L@@@(..)
          testQueryEndKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@]O@^@L@@@^A^K^@A@@@Ohttp://bar.com/^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Ohttp://bar.com/(..)
          testQueryKeyRange(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@[O@^@L@@@^A^K^@A@@@Ohttp://bar.com/^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Ohttp://bar.com/(..)
          testQueryWebPageSingleKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80>A@B@@@Pget_range_slices@@@j^O^@^@L@@@^A^K^@A@@@Ohttp://foo.com/^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Ohttp://foo.com/(..)
          testDelete(org.apache.gora.cassandra.store.TestCassandraStore)

          Tests in error:
          testGetWebPageDefaultFields(org.apache.gora.cassandra.store.TestCassandraStore)
          testQueryWebPageSingleKeyDefaultFields(org.apache.gora.cassandra.store.TestCassandraStore)
          testDeleteByQuery(org.apache.gora.cassandra.store.TestCassandraStore)
          testDeleteByQueryFields(org.apache.gora.cassandra.store.TestCassandraStore)
          testGetPartitions(org.apache.gora.cassandra.store.TestCassandraStore)

          Tests run: 28, Failures: 9, Errors: 5, Skipped: 0

          [ERROR] There are test failures.

          Show
          Kazuomi Kashii added a comment - Finally, gora-cassandra tests started working after some additional changes. 1) Applied GORA-148 , GORA-142 , and GORA-149 patches. 2) Uses cassandra-all 1.1.0 (not 1.1.1) 3) gora-cassandra/src/test/conf/gora-cassandra-mapping.xml keyspace: WebPage -> webpage family="sc" for parsedContent, outlinks, and metadata Tests run: 28, Failures: 9, Errors: 5, Skipped: 0 Results : Failed tests: testSchemaExists(org.apache.gora.cassandra.store.TestCassandraStore) testUpdate(org.apache.gora.cassandra.store.TestCassandraStore): expected:< [content0] > but was:<[? A @ B @ @ @ Pget_range_slices @ @ @5^O^@^@ L @ @ @^A^K^@ A @ @ @ Nhttp://a.com/a^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Nhttp://a.com/a(..) testGetWebPage(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @= O @^@ L @ @ @^A^K^@ A @ @ @ Ohttp://foo.com/^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Ohttp://foo.com/(..) testQuery(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @O^O^@^@ L @ @ @(..) testQueryStartKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @Q^O^@^@ L @ @ @(..) testQueryEndKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @] O @^@ L @ @ @^A^K^@ A @ @ @ Ohttp://bar.com/^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Ohttp://bar.com/(..) testQueryKeyRange(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @[ O @^@ L @ @ @^A^K^@ A @ @ @ Ohttp://bar.com/^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Ohttp://bar.com/(..) testQueryWebPageSingleKey(org.apache.gora.cassandra.store.TestCassandraStore): content error:<80> A @ B @ @ @ Pget_range_slices @ @ @j^O^@^@ L @ @ @^A^K^@ A @ @ @ Ohttp://foo.com/^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Ohttp://foo.com/(..) testDelete(org.apache.gora.cassandra.store.TestCassandraStore) Tests in error: testGetWebPageDefaultFields(org.apache.gora.cassandra.store.TestCassandraStore) testQueryWebPageSingleKeyDefaultFields(org.apache.gora.cassandra.store.TestCassandraStore) testDeleteByQuery(org.apache.gora.cassandra.store.TestCassandraStore) testDeleteByQueryFields(org.apache.gora.cassandra.store.TestCassandraStore) testGetPartitions(org.apache.gora.cassandra.store.TestCassandraStore) Tests run: 28, Failures: 9, Errors: 5, Skipped: 0 [ERROR] There are test failures.
          Hide
          Kazuomi Kashii added a comment -

          The dependency issue of avro-1.4.0-cassandra-1.jar can be solved with exclusion in pom.xml as follows:

          <dependency>
          <groupId>org.apache.cassandra</groupId>
          <artifactId>cassandra-all</artifactId>
          <exclusions>
          <exclusion>
          <groupId>org.apache.cassandra.deps</groupId>
          <artifactId>avro</artifactId>
          </exclusion>
          </exclusions>
          </dependency>

          Show
          Kazuomi Kashii added a comment - The dependency issue of avro-1.4.0-cassandra-1.jar can be solved with exclusion in pom.xml as follows: <dependency> <groupId>org.apache.cassandra</groupId> <artifactId>cassandra-all</artifactId> <exclusions> <exclusion> <groupId>org.apache.cassandra.deps</groupId> <artifactId>avro</artifactId> </exclusion> </exclusions> </dependency>
          Hide
          Kazuomi Kashii added a comment -

          Lewis, "too many open files" error is a well known issue of Cassandra.
          We need to adjust the OS parameter(s) to avoid this kind of erros.
          http://www.datastax.com/docs/1.0/troubleshooting/index#toomany

          Also, we may be able to adjust setUp() and tearDown() methods.

          Show
          Kazuomi Kashii added a comment - Lewis, "too many open files" error is a well known issue of Cassandra. We need to adjust the OS parameter(s) to avoid this kind of erros. http://www.datastax.com/docs/1.0/troubleshooting/index#toomany Also, we may be able to adjust setUp() and tearDown() methods.
          Hide
          Kazuomi Kashii added a comment -

          Tests run: 28, Failures: 9, Errors: 5, Skipped: 0, Time elapsed: 21.147 sec <<< FAILURE!

          There are three types of failures or errors.

          1) AssertionFailedError

          testSchemaExists(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.883 sec <<< FAILURE!
          junit.framework.AssertionFailedError
          at junit.framework.Assert.fail(Assert.java:48)
          at junit.framework.Assert.assertTrue(Assert.java:20)
          at junit.framework.Assert.assertTrue(Assert.java:27)
          at org.apache.gora.store.DataStoreTestUtil.testSchemaExists(DataStoreTestUtil.java:131)
          at org.apache.gora.store.DataStoreTestBase.testSchemaExists(DataStoreTestBase.java:164)

          It is because CassandraStore's schemaExists() method returns always false.

          public boolean schemaExists() throws IOException

          { LOG.info("schema exists"); return false; }

          2) StackOverflowError

          testDeleteByQuery(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.62 sec <<< ERROR!
          java.lang.StackOverflowError
          at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32)
          at org.apache.avro.generic.GenericData.compare(GenericData.java:487)
          at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:102)
          at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32)
          at org.apache.avro.generic.GenericData.compare(GenericData.java:487)
          at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:102)
          at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32)
          at org.apache.avro.generic.GenericData.compare(GenericData.java:487)

          I have no clue. Maybe a bug of gora-core ?

          3) ComparisonFailure

          testUpdate(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.68 sec <<< FAILURE!
          junit.framework.ComparisonFailure: expected:<[content0]> but was:<[?A@B@@@Pget_range_slices@@@5^O^@^@L@@@^A^K^@A@@@Nhttp://a.com/a^O@^B^L^@@@^B^L^@^A^K^@A@@@Cc:u^K@B@@@^Nhttp://a.com/a
          ^@C@D????5?@^@L@^A^K^@A@@@Gp:cnt:c^K@B@@@^Hcontent0
          ^@C@D????+?@@@^@]>
          at junit.framework.Assert.assertEquals(Assert.java:85)
          at junit.framework.Assert.assertEquals(Assert.java:91)
          at org.apache.gora.store.DataStoreTestUtil.testUpdateWebPage(DataStoreTestUtil.java:296)
          at org.apache.gora.store.DataStoreTestBase.testUpdate(DataStoreTestBase.java:267)

          It seems a bug of gora-cassandra.

          Show
          Kazuomi Kashii added a comment - Tests run: 28, Failures: 9, Errors: 5, Skipped: 0, Time elapsed: 21.147 sec <<< FAILURE! There are three types of failures or errors. 1) AssertionFailedError testSchemaExists(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.883 sec <<< FAILURE! junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:48) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.gora.store.DataStoreTestUtil.testSchemaExists(DataStoreTestUtil.java:131) at org.apache.gora.store.DataStoreTestBase.testSchemaExists(DataStoreTestBase.java:164) It is because CassandraStore's schemaExists() method returns always false. public boolean schemaExists() throws IOException { LOG.info("schema exists"); return false; } 2) StackOverflowError testDeleteByQuery(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.62 sec <<< ERROR! java.lang.StackOverflowError at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32) at org.apache.avro.generic.GenericData.compare(GenericData.java:487) at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:102) at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32) at org.apache.avro.generic.GenericData.compare(GenericData.java:487) at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:102) at org.apache.gora.persistency.ListGenericArray.compareTo(ListGenericArray.java:32) at org.apache.avro.generic.GenericData.compare(GenericData.java:487) I have no clue. Maybe a bug of gora-core ? 3) ComparisonFailure testUpdate(org.apache.gora.cassandra.store.TestCassandraStore) Time elapsed: 0.68 sec <<< FAILURE! junit.framework.ComparisonFailure: expected:< [content0] > but was:<[? A @ B @ @ @ Pget_range_slices @ @ @5^O^@^@ L @ @ @^A^K^@ A @ @ @ Nhttp://a.com/a^O @^B^L^@ @ @^B^L^@^A^K^@ A @ @ @ Cc:u^K @ B @ @ @^Nhttp://a.com/a ^@ C @ D????5? @^@ L @^A^K^@ A @ @ @ Gp:cnt:c^K @ B @ @ @^Hcontent0 ^@ C @ D????+? @ @ @^@]> at junit.framework.Assert.assertEquals(Assert.java:85) at junit.framework.Assert.assertEquals(Assert.java:91) at org.apache.gora.store.DataStoreTestUtil.testUpdateWebPage(DataStoreTestUtil.java:296) at org.apache.gora.store.DataStoreTestBase.testUpdate(DataStoreTestBase.java:267) It seems a bug of gora-cassandra.
          Hide
          Kazuomi Kashii added a comment -

          GORA-test.patch - GORA-142 - GORA-148 - GORA-149 + some additional fixes

          Show
          Kazuomi Kashii added a comment - GORA-test.patch - GORA-142 - GORA-148 - GORA-149 + some additional fixes
          Hide
          Kazuomi Kashii added a comment -

          $ svn commit --username kazk --message "Commits GORA-53 final patch"
          Sending gora-cassandra/pom.xml
          Sending gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java
          Adding gora-cassandra/src/test/conf/cassandra.yaml
          Sending gora-cassandra/src/test/conf/gora-cassandra-mapping.xml
          Sending gora-cassandra/src/test/conf/gora.properties
          Adding gora-cassandra/src/test/conf/log4j-server.properties
          Adding gora-cassandra/src/test/java/org
          Adding gora-cassandra/src/test/java/org/apache
          Adding gora-cassandra/src/test/java/org/apache/gora
          Adding gora-cassandra/src/test/java/org/apache/gora/cassandra
          Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java
          Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/store
          Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java
          Sending gora-core/src/main/java/overview.html
          Sending gora-core/src/test/java/org/apache/gora/GoraTestDriver.java
          Sending gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java
          Sending gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java
          Sending gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java
          Sending gora-sql/pom.xml
          Sending pom.xml
          Transmitting file data ...............
          Committed revision 1362989.
          $

          Show
          Kazuomi Kashii added a comment - $ svn commit --username kazk --message "Commits GORA-53 final patch" Sending gora-cassandra/pom.xml Sending gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java Adding gora-cassandra/src/test/conf/cassandra.yaml Sending gora-cassandra/src/test/conf/gora-cassandra-mapping.xml Sending gora-cassandra/src/test/conf/gora.properties Adding gora-cassandra/src/test/conf/log4j-server.properties Adding gora-cassandra/src/test/java/org Adding gora-cassandra/src/test/java/org/apache Adding gora-cassandra/src/test/java/org/apache/gora Adding gora-cassandra/src/test/java/org/apache/gora/cassandra Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/store Adding gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java Sending gora-core/src/main/java/overview.html Sending gora-core/src/test/java/org/apache/gora/GoraTestDriver.java Sending gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java Sending gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java Sending gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java Sending gora-sql/pom.xml Sending pom.xml Transmitting file data ............... Committed revision 1362989. $
          Hide
          Hudson added a comment -

          Integrated in gora-trunk #353 (See https://builds.apache.org/job/gora-trunk/353/)
          Commits GORA-53 final patch (Revision 1362989)

          Result = UNSTABLE
          kazk :
          Files :

          • /gora/trunk/gora-cassandra/pom.xml
          • /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java
          • /gora/trunk/gora-cassandra/src/test/conf/cassandra.yaml
          • /gora/trunk/gora-cassandra/src/test/conf/gora-cassandra-mapping.xml
          • /gora/trunk/gora-cassandra/src/test/conf/gora.properties
          • /gora/trunk/gora-cassandra/src/test/conf/log4j-server.properties
          • /gora/trunk/gora-cassandra/src/test/java/org
          • /gora/trunk/gora-cassandra/src/test/java/org/apache
          • /gora/trunk/gora-cassandra/src/test/java/org/apache/gora
          • /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra
          • /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java
          • /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/store
          • /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java
          • /gora/trunk/gora-core/src/main/java/overview.html
          • /gora/trunk/gora-core/src/test/java/org/apache/gora/GoraTestDriver.java
          • /gora/trunk/gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java
          • /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java
          • /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java
          • /gora/trunk/gora-sql/pom.xml
          • /gora/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in gora-trunk #353 (See https://builds.apache.org/job/gora-trunk/353/ ) Commits GORA-53 final patch (Revision 1362989) Result = UNSTABLE kazk : Files : /gora/trunk/gora-cassandra/pom.xml /gora/trunk/gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java /gora/trunk/gora-cassandra/src/test/conf/cassandra.yaml /gora/trunk/gora-cassandra/src/test/conf/gora-cassandra-mapping.xml /gora/trunk/gora-cassandra/src/test/conf/gora.properties /gora/trunk/gora-cassandra/src/test/conf/log4j-server.properties /gora/trunk/gora-cassandra/src/test/java/org /gora/trunk/gora-cassandra/src/test/java/org/apache /gora/trunk/gora-cassandra/src/test/java/org/apache/gora /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/store /gora/trunk/gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java /gora/trunk/gora-core/src/main/java/overview.html /gora/trunk/gora-core/src/test/java/org/apache/gora/GoraTestDriver.java /gora/trunk/gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java /gora/trunk/gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java /gora/trunk/gora-sql/pom.xml /gora/trunk/pom.xml
          Hide
          Lewis John McGibbney added a comment -

          Committed @r1362989 in trunk
          Thanks Kaz for all the hard work.

          Show
          Lewis John McGibbney added a comment - Committed @r1362989 in trunk Thanks Kaz for all the hard work.
          Hide
          Hudson added a comment -

          Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/)
          Commits GORA-53 final patch (Revision 7782f6e6d4336c55de08ac73637b889b3feb499c)

          Result = FAILURE
          kazk :
          Files :

          • pom.xml
          • gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java
          • gora-cassandra/pom.xml
          • gora-core/src/main/java/overview.html
          • gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java
          • gora-sql/pom.xml
          • gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java
          • gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java
          • gora-cassandra/src/test/conf/gora-cassandra-mapping.xml
          • gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java
          • gora-cassandra/src/test/conf/cassandra.yaml
          • gora-core/src/test/java/org/apache/gora/GoraTestDriver.java
          • gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java
          • gora-cassandra/src/test/conf/gora.properties
          • gora-cassandra/src/test/conf/log4j-server.properties
          Show
          Hudson added a comment - Integrated in goraOracle #6 (See https://builds.apache.org/job/goraOracle/6/ ) Commits GORA-53 final patch (Revision 7782f6e6d4336c55de08ac73637b889b3feb499c) Result = FAILURE kazk : Files : pom.xml gora-hbase/src/test/java/org/apache/gora/hbase/GoraHBaseTestDriver.java gora-cassandra/pom.xml gora-core/src/main/java/overview.html gora-core/src/test/java/org/apache/gora/store/DataStoreTestBase.java gora-sql/pom.xml gora-cassandra/src/test/java/org/apache/gora/cassandra/GoraCassandraTestDriver.java gora-cassandra/src/test/java/org/apache/gora/cassandra/store/TestCassandraStore.java gora-cassandra/src/test/conf/gora-cassandra-mapping.xml gora-hbase/src/test/java/org/apache/gora/hbase/store/TestHBaseStore.java gora-cassandra/src/test/conf/cassandra.yaml gora-core/src/test/java/org/apache/gora/GoraTestDriver.java gora-cassandra/src/main/java/org/apache/gora/cassandra/store/CassandraMappingManager.java gora-cassandra/src/test/conf/gora.properties gora-cassandra/src/test/conf/log4j-server.properties

            People

            • Assignee:
              Lewis John McGibbney
              Reporter:
              Lewis John McGibbney
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development