Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: nutchgora
    • Fix Version/s: nutchgora
    • Component/s: storage
    • Labels:
      None

      Description

      This issue is part of a larger target which aims to fix broken JUnit tests for Nutchgora

      1. NUTCH-1135-v1.patch
        10 kB
        Ferdy Galema
      2. NUTCH-1135-v2.patch
        10 kB
        Ferdy Galema
      3. NUTCH-1135-v3.patch
        14 kB
        Ferdy Galema
      4. NUTCH-1135-v4.patch
        14 kB
        Ferdy Galema

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Nutch-nutchgora #39 (See https://builds.apache.org/job/Nutch-nutchgora/39/)
          commit to address NUTCH-1135 and update to changes.txt

          lewismc : http://svn.apache.org/viewvc/nutch/branches/nutchgora/viewvc/?view=rev&root=&revision=1186766
          Files :

          • /nutch/branches/nutchgora/CHANGES.txt
          • /nutch/branches/nutchgora/src/test/org/apache/nutch/storage/TestGoraStorage.java
          Show
          Hudson added a comment - Integrated in Nutch-nutchgora #39 (See https://builds.apache.org/job/Nutch-nutchgora/39/ ) commit to address NUTCH-1135 and update to changes.txt lewismc : http://svn.apache.org/viewvc/nutch/branches/nutchgora/viewvc/?view=rev&root=&revision=1186766 Files : /nutch/branches/nutchgora/CHANGES.txt /nutch/branches/nutchgora/src/test/org/apache/nutch/storage/TestGoraStorage.java
          Hide
          Lewis John McGibbney added a comment -

          Committed @ revision 1186766 in nutchgora branch

          Further to additional comments, this issue can be reopened and the test edited or additional test cases added, however at this stage the test provides a substantial move towards ensuring a more stable branch.

          Show
          Lewis John McGibbney added a comment - Committed @ revision 1186766 in nutchgora branch Further to additional comments, this issue can be reopened and the test edited or additional test cases added, however at this stage the test provides a substantial move towards ensuring a more stable branch.
          Hide
          Ferdy Galema added a comment -

          If there are no further objections to committing the latest patch I'd say go for it.

          Show
          Ferdy Galema added a comment - If there are no further objections to committing the latest patch I'd say go for it.
          Hide
          Lewis John McGibbney added a comment -

          Hi Andrzej do you have any comments (or anyone else for that matter) on Ferdy's v4 patch? Having tested the patch I can confirm that runtime for this test is by far the longest for any of nutchgora existing tests, this of course can be attributed to his decision to retain and revise testMultiProcess. Some pretty comprehensive test output can be seen below

          lewis@lewis-01:~/ASF/nutchgora/runtime/local$ bin/nutch junit org.apache.nutch.storage.TestGoraStorage
          ...[Server@1ef4b31]: [Thread[main,5,main]]: checkRunning(false) entered
          [Server@1ef4b31]: [Thread[main,5,main]]: checkRunning(false) exited
          [Server@1ef4b31]: [Thread[main,5,main]]: setDaemon(true)
          [Server@1ef4b31]: [Thread[main,5,main]]: setSilent(true)
          [Server@1ef4b31]: Initiating startup sequence...
          [Server@1ef4b31]: Server socket opened successfully in 38 ms.
          [Server@1ef4b31]: Database [index=0, id=0, db=mem:org.apache.nutch.storage.testgorastorage, alias=org.apache.nutch.storage.testgorastorage] opened sucessfully in 0 ms.
          [Server@1ef4b31]: Startup sequence completed in 39 ms.
          [Server@1ef4b31]: 2011-10-18 17:18:03.852 HSQLDB server 2.2.4 is online on port 9001
          [Server@1ef4b31]: To close normally, connect and execute SHUTDOWN SQL
          [Server@1ef4b31]: From command line, use [Ctrl]+[C] to abort abruptly
          ===Process stream for Thread[pool-2-thread-3,5,main]
          Starting!
          Done.
          ===End of process stream.
          ===Process stream for Thread[pool-2-thread-1,5,main]
          Starting!
          Done.
          ===End of process stream.
          ===Process stream for Thread[pool-2-thread-4,5,main]
          Starting!
          Done.
          ===End of process stream.
          ===Process stream for Thread[pool-2-thread-2,5,main]
          Starting!
          Done.
          ===End of process stream.
          [Server@1ef4b31]: Initiating shutdown sequence...
          [Server@1ef4b31]: Shutdown sequence completed in 0 ms.
          [Server@1ef4b31]: 2011-10-18 17:20:10.394 SHUTDOWN : System.exit() was not called
          
          Time: 142.643
          
          OK (3 tests)
          
          

          I am pretty convinced that the test satisfies your points made regarding map-reduce processes.
          I think a nice test would be to query a db for test data to ensure this are working nicely on the gora-side, however as was mentioned previously, I think this falls outside the scope of nutchgora and more directly on Gora itself.

          I cna only shadow Andrzej's comments, thanks Ferdy for your hard work here. I think were making great progress with this and look forward to seeing this issue resolved.

          Show
          Lewis John McGibbney added a comment - Hi Andrzej do you have any comments (or anyone else for that matter) on Ferdy's v4 patch? Having tested the patch I can confirm that runtime for this test is by far the longest for any of nutchgora existing tests, this of course can be attributed to his decision to retain and revise testMultiProcess. Some pretty comprehensive test output can be seen below lewis@lewis-01:~/ASF/nutchgora/runtime/local$ bin/nutch junit org.apache.nutch.storage.TestGoraStorage ...[Server@1ef4b31]: [ Thread [main,5,main]]: checkRunning( false ) entered [Server@1ef4b31]: [ Thread [main,5,main]]: checkRunning( false ) exited [Server@1ef4b31]: [ Thread [main,5,main]]: setDaemon( true ) [Server@1ef4b31]: [ Thread [main,5,main]]: setSilent( true ) [Server@1ef4b31]: Initiating startup sequence... [Server@1ef4b31]: Server socket opened successfully in 38 ms. [Server@1ef4b31]: Database [index=0, id=0, db=mem:org.apache.nutch.storage.testgorastorage, alias=org.apache.nutch.storage.testgorastorage] opened sucessfully in 0 ms. [Server@1ef4b31]: Startup sequence completed in 39 ms. [Server@1ef4b31]: 2011-10-18 17:18:03.852 HSQLDB server 2.2.4 is online on port 9001 [Server@1ef4b31]: To close normally, connect and execute SHUTDOWN SQL [Server@1ef4b31]: From command line, use [Ctrl]+[C] to abort abruptly === Process stream for Thread [pool-2-thread-3,5,main] Starting! Done. ===End of process stream. === Process stream for Thread [pool-2-thread-1,5,main] Starting! Done. ===End of process stream. === Process stream for Thread [pool-2-thread-4,5,main] Starting! Done. ===End of process stream. === Process stream for Thread [pool-2-thread-2,5,main] Starting! Done. ===End of process stream. [Server@1ef4b31]: Initiating shutdown sequence... [Server@1ef4b31]: Shutdown sequence completed in 0 ms. [Server@1ef4b31]: 2011-10-18 17:20:10.394 SHUTDOWN : System .exit() was not called Time: 142.643 OK (3 tests) I am pretty convinced that the test satisfies your points made regarding map-reduce processes. I think a nice test would be to query a db for test data to ensure this are working nicely on the gora-side, however as was mentioned previously, I think this falls outside the scope of nutchgora and more directly on Gora itself. I cna only shadow Andrzej's comments, thanks Ferdy for your hard work here. I think were making great progress with this and look forward to seeing this issue resolved.
          Hide
          Ferdy Galema added a comment -

          Latest patch contains minor fixes. (Some comments were incorrect.)

          Show
          Ferdy Galema added a comment - Latest patch contains minor fixes. (Some comments were incorrect.)
          Hide
          Ferdy Galema added a comment -

          New patch with multi process test done.

          Creates an on-the-fly hsql server with multiple connecting clients. Double checked with a non-process safe implementation (jdbc:hsqldb:file) and this fails the test (cannot acquire lock errors).

          Using a stand-alone server works perfectly.

          Show
          Ferdy Galema added a comment - New patch with multi process test done. Creates an on-the-fly hsql server with multiple connecting clients. Double checked with a non-process safe implementation (jdbc:hsqldb:file) and this fails the test (cannot acquire lock errors). Using a stand-alone server works perfectly.
          Hide
          Andrzej Bialecki added a comment -

          if you prefer to keep the old TestGoraStorage structure

          Not really, I'm not against cleanup / breaking it up - if it makes sense let's go for it. My main concern was that by skipping the multi-process test altogether we would ignore testing a part of Gora functionality that is critical to Nutch (well, to any other map-reduce app, too, but we're doing Nutch here ).

          Thank you for your persistence.

          By the way, I tested the testMultithreaded with a DataStore that is not thread safe

          Excellent!

          Show
          Andrzej Bialecki added a comment - if you prefer to keep the old TestGoraStorage structure Not really, I'm not against cleanup / breaking it up - if it makes sense let's go for it. My main concern was that by skipping the multi-process test altogether we would ignore testing a part of Gora functionality that is critical to Nutch (well, to any other map-reduce app, too, but we're doing Nutch here ). Thank you for your persistence. By the way, I tested the testMultithreaded with a DataStore that is not thread safe Excellent!
          Hide
          Ferdy Galema added a comment -

          By the way, I tested the testMultithreaded with a DataStore that is not thread safe (namely an implementation with simply HashMap as a backing) to verify it is testing thread-safeness correctly, and I can state that it does. Failures are almost guaranteed when running the test with this 'dummy' implementation.

          Show
          Ferdy Galema added a comment - By the way, I tested the testMultithreaded with a DataStore that is not thread safe (namely an implementation with simply HashMap as a backing) to verify it is testing thread-safeness correctly, and I can state that it does. Failures are almost guaranteed when running the test with this 'dummy' implementation.
          Hide
          Ferdy Galema added a comment -

          Thank you for responding. The purpose of the multi-process test is clear to me now and I fully agree with your arguments.

          I will reinstate a similar test in the new implementation and try to get it working properly. (If that's okay with you; if you prefer to keep the old TestGoraStorage structure please let me know and I'd be more than happy to delegate the task to you).

          In short, a new patch is in progress!

          Show
          Ferdy Galema added a comment - Thank you for responding. The purpose of the multi-process test is clear to me now and I fully agree with your arguments. I will reinstate a similar test in the new implementation and try to get it working properly. (If that's okay with you; if you prefer to keep the old TestGoraStorage structure please let me know and I'd be more than happy to delegate the task to you). In short, a new patch is in progress!
          Hide
          Andrzej Bialecki added a comment -

          A few comments from the author of this monstrosity First, thanks Ferdy for taking time to work with this, it's much appreciated, we need to move forward on this. I agree that ultimately this test should be moved to Gora and become a part of a larger test suite that verifies correctness of concurrent multi-threaded and multi-process operations.

          However, the immediate purpose of this class was to stress-test the existing Gora versions in usage patterns typical for Nutch, in order to verify that a particular version of Gora is a viable storage layer for Nutch - so the test tries to replicate typical Nutch scenarios. Remember that this has to work not only for a toy crawl in a single JVM in local mode, but also for a fully distributed parallel map-reduce crawl. Consequently:

          • testMultiThread: tests a scenario of multiple threads in a single JVM all writing to the same storage instance. This replicates a scenario present e.g. in a single Fetcher task. If this test fails (assuming it's properly constructed!) then this means that Gora will fail, perhaps silently (see NUTCH-893), in a fundamental Nutch tool.
          • testMultiProcess: tests a scenario of multiple processes running in multiple JVMs all writing to the same storage instance. This replicates a scenario of multiple map-reduce tasks all using the same storage config (shared storage, e.g. HSQLDB in server mode), and it's fundamental to all Nutch tools running on a cluster. In map-reduce jobs there are usually many concurrent tasks, and some of them may execute in several copies in parallel (speculative execution) and some others may fail catastrophically without proper cleanup - and Gora backends must just deal with it. If this test fails (again, assuming it's properly constructed and doesn't exceed some OS capabilities of the test machine, or some known limits of a storage impl. like the number of concurrent connections) then it means that Gora storage is not reliable for a typical map-reduce usage, which sort of defeats the point of using it at all.

          To summarize: I think the patch in its current form helps the tests pass, but I don't think it addresses the underlying problems in Gora (or perhaps the problems with HSQL backend), rather it hides the problem. After all, we want the test to mean something if it passes, to verify that we can use Gora for more than a toy crawl, with guarantees of correctness in presence of concurrent updates.

          If the above errors don't indicate issues with Gora, but instead are caused by exceeded OS or hsql limits, or hsql misconfiguration, then of course we should fix the configs and adjust the numbers so that they make sense. But with the proper config and proper numbers both tests should pass, otherwise we can't be sure that Gora is working properly at all.

          Show
          Andrzej Bialecki added a comment - A few comments from the author of this monstrosity First, thanks Ferdy for taking time to work with this, it's much appreciated, we need to move forward on this. I agree that ultimately this test should be moved to Gora and become a part of a larger test suite that verifies correctness of concurrent multi-threaded and multi-process operations. However, the immediate purpose of this class was to stress-test the existing Gora versions in usage patterns typical for Nutch, in order to verify that a particular version of Gora is a viable storage layer for Nutch - so the test tries to replicate typical Nutch scenarios. Remember that this has to work not only for a toy crawl in a single JVM in local mode, but also for a fully distributed parallel map-reduce crawl. Consequently: testMultiThread: tests a scenario of multiple threads in a single JVM all writing to the same storage instance. This replicates a scenario present e.g. in a single Fetcher task. If this test fails (assuming it's properly constructed!) then this means that Gora will fail, perhaps silently (see NUTCH-893 ), in a fundamental Nutch tool. testMultiProcess: tests a scenario of multiple processes running in multiple JVMs all writing to the same storage instance. This replicates a scenario of multiple map-reduce tasks all using the same storage config (shared storage, e.g. HSQLDB in server mode), and it's fundamental to all Nutch tools running on a cluster. In map-reduce jobs there are usually many concurrent tasks, and some of them may execute in several copies in parallel (speculative execution) and some others may fail catastrophically without proper cleanup - and Gora backends must just deal with it. If this test fails (again, assuming it's properly constructed and doesn't exceed some OS capabilities of the test machine, or some known limits of a storage impl. like the number of concurrent connections) then it means that Gora storage is not reliable for a typical map-reduce usage, which sort of defeats the point of using it at all. To summarize: I think the patch in its current form helps the tests pass, but I don't think it addresses the underlying problems in Gora (or perhaps the problems with HSQL backend), rather it hides the problem. After all, we want the test to mean something if it passes, to verify that we can use Gora for more than a toy crawl, with guarantees of correctness in presence of concurrent updates. If the above errors don't indicate issues with Gora, but instead are caused by exceeded OS or hsql limits, or hsql misconfiguration, then of course we should fix the configs and adjust the numbers so that they make sense. But with the proper config and proper numbers both tests should pass, otherwise we can't be sure that Gora is working properly at all.
          Hide
          Lewis John McGibbney added a comment -

          OK. I think in general we are not keen on dropping any code as such, however in this case I think the benefits outweigh any other factors as this would enable us to get a stable branch back on track after around 13 odd months! I would however like to wait until we can get feedback from at least one other committer before progressing. We have waited quite long, so an extra week or so will not hurt us. Thanks Ferdy

          Show
          Lewis John McGibbney added a comment - OK. I think in general we are not keen on dropping any code as such, however in this case I think the benefits outweigh any other factors as this would enable us to get a stable branch back on track after around 13 odd months! I would however like to wait until we can get feedback from at least one other committer before progressing. We have waited quite long, so an extra week or so will not hurt us. Thanks Ferdy
          Hide
          Ferdy Galema added a comment -

          No problem, I think we all benefit from a stable Nutchgora branch

          You are right about the indent style, I was using tabs instead of spaces. Latest patch uses standard Nutch 2-spaces indent.

          Show
          Ferdy Galema added a comment - No problem, I think we all benefit from a stable Nutchgora branch You are right about the indent style, I was using tabs instead of spaces. Latest patch uses standard Nutch 2-spaces indent.
          Hide
          Lewis John McGibbney added a comment -

          First and foremost, I must say a personal big thank you, and I'm sure I speak on behalf of the dev community as well. Having looked at you proposed code, I think it is definitely beneficial to include the testSinglethreaded test. It gives the testGoraStorage a fluid nature and provides nice consistency. The comments are excellent, exactly the type of thing we require. I would comment though that the indent style is somewhat off when drawing comparison with the rest of the nuctchgora codebase, would it be possible for you to edit this please and resubmit a patch? If you could get this sorted, then it would be great if some other folks could have a look and pass comments. I was fairly new to the nutchgora test suites and have also been learning as we go along. The delightful sight of nutchgora finally passing all test and building nearly brought a tear to my eye :0)

          Show
          Lewis John McGibbney added a comment - First and foremost, I must say a personal big thank you, and I'm sure I speak on behalf of the dev community as well. Having looked at you proposed code, I think it is definitely beneficial to include the testSinglethreaded test. It gives the testGoraStorage a fluid nature and provides nice consistency. The comments are excellent, exactly the type of thing we require. I would comment though that the indent style is somewhat off when drawing comparison with the rest of the nuctchgora codebase, would it be possible for you to edit this please and resubmit a patch? If you could get this sorted, then it would be great if some other folks could have a look and pass comments. I was fairly new to the nutchgora test suites and have also been learning as we go along. The delightful sight of nutchgora finally passing all test and building nearly brought a tear to my eye :0)
          Hide
          Ferdy Galema added a comment -

          This patch is a rewrite of the test. I thought it was the right thing to do because some parts were really overcomplicated. The major changes:
          -Dropped the multiprocess test. Could not get it to work, even with conservative number of processes (processes exit with nonzero exitcode). Also what would be the purpose of such a test? A shared DataStore should be used anyway when the processes want to share webpages. Perhaps the scope of such a test is outside Nutch. The Gora project itself would be a better place for it.
          -TestGoraStorage extends AbstractNutchTest now.
          -More simple code and comments.

          If anyone has comments or a different view on this, please let me know.

          Show
          Ferdy Galema added a comment - This patch is a rewrite of the test. I thought it was the right thing to do because some parts were really overcomplicated. The major changes: -Dropped the multiprocess test. Could not get it to work, even with conservative number of processes (processes exit with nonzero exitcode). Also what would be the purpose of such a test? A shared DataStore should be used anyway when the processes want to share webpages. Perhaps the scope of such a test is outside Nutch. The Gora project itself would be a better place for it. -TestGoraStorage extends AbstractNutchTest now. -More simple code and comments. If anyone has comments or a different view on this, please let me know.
          Hide
          Ferdy Galema added a comment -

          No problem I'll work out a patch that fixes the test (at least the first part) and, while I'm at it, document a few things here and there. Should be ready in a few days.

          Show
          Ferdy Galema added a comment - No problem I'll work out a patch that fixes the test (at least the first part) and, while I'm at it, document a few things here and there. Should be ready in a few days.
          Hide
          Lewis John McGibbney added a comment -

          Hi Ferdy, firstly thanks for looking in to this. In reply I would initially ask if it is possible for you to attach a patch which satisfies

          "With this the first part of the test succeeds (threads test)"

          . Once we get this working we can progress with possibly editing/commenting/removing other parts of the test. Unfortunately when this was committed, it was not documented, I think it would be great as our contribution if we could get a comprehensively documented comment added it would be twice as useful. The reason for a staggered approach to getting this sorted is that, it has become obvious to me over time that we need to ensure nutchgora is JUnit compliant as much as possible. Only when this requisite, plus fully documented classes and additional resources are in place, is there grounds for a stable first release. Thanks Ferdy

          Show
          Lewis John McGibbney added a comment - Hi Ferdy, firstly thanks for looking in to this. In reply I would initially ask if it is possible for you to attach a patch which satisfies "With this the first part of the test succeeds (threads test)" . Once we get this working we can progress with possibly editing/commenting/removing other parts of the test. Unfortunately when this was committed, it was not documented, I think it would be great as our contribution if we could get a comprehensively documented comment added it would be twice as useful. The reason for a staggered approach to getting this sorted is that, it has become obvious to me over time that we need to ensure nutchgora is JUnit compliant as much as possible. Only when this requisite, plus fully documented classes and additional resources are in place, is there grounds for a stable first release. Thanks Ferdy
          Hide
          Ferdy Galema added a comment -

          It seems like Gora simply tries to connect to a non-existing sql service. Setting a memory sql store (as AbstractNutchTest does) seems like a quick fix.

          With this the first part of the test succeeds (threads test) but my computer is not able to finish the "spawning processes" test. It simply crashes. It is truly a monstrous test (I now see what you mean with "the demon that is TestGoraStorage" ...).

          Anyway, I propose to simplify (remove?) the second test. What are your thoughts on this?

          Show
          Ferdy Galema added a comment - It seems like Gora simply tries to connect to a non-existing sql service. Setting a memory sql store (as AbstractNutchTest does) seems like a quick fix. With this the first part of the test succeeds (threads test) but my computer is not able to finish the "spawning processes" test. It simply crashes. It is truly a monstrous test (I now see what you mean with "the demon that is TestGoraStorage" ...). Anyway, I propose to simplify (remove?) the second test. What are your thoughts on this?
          Hide
          Lewis John McGibbney added a comment -

          full stack trace when attempting to run this test case

          lewis@lewis-01:~/ASF/nutchgora/runtime/local$ bin/nutch junit org.apache.nutch.storage.TestGoraStorage
          .E.E
          Time: 0.931
          There were 2 errors:
          1) testMultithread(org.apache.nutch.storage.TestGoraStorage)org.apache.gora.util.GoraException: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:110)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:87)
          	at org.apache.nutch.storage.StorageUtils.createDataStore(StorageUtils.java:43)
          	at org.apache.nutch.storage.TestGoraStorage.setUp(TestGoraStorage.java:47)
          Caused by: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:747)
          	at org.apache.gora.sql.store.SqlStore.initialize(SqlStore.java:160)
          	at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:81)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:104)
          	... 14 more
          Caused by: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
          	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
          	at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source)
          	at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
          	at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
          	at java.sql.DriverManager.getConnection(DriverManager.java:582)
          	at java.sql.DriverManager.getConnection(DriverManager.java:207)
          	at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:737)
          	... 17 more
          Caused by: org.hsqldb.HsqlException: java.net.ConnectException: Connection refused
          	at org.hsqldb.ClientConnection.openConnection(Unknown Source)
          	at org.hsqldb.ClientConnection.initConnection(Unknown Source)
          	at org.hsqldb.ClientConnection.<init>(Unknown Source)
          	... 23 more
          Caused by: java.net.ConnectException: Connection refused
          	at java.net.PlainSocketImpl.socketConnect(Native Method)
          	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
          	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
          	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
          	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
          	at java.net.Socket.connect(Socket.java:529)
          	at java.net.Socket.connect(Socket.java:478)
          	at java.net.Socket.<init>(Socket.java:375)
          	at java.net.Socket.<init>(Socket.java:189)
          	at org.hsqldb.server.HsqlSocketFactory.createSocket(Unknown Source)
          	... 26 more
          2) testMultiProcess(org.apache.nutch.storage.TestGoraStorage)org.apache.gora.util.GoraException: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:110)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:87)
          	at org.apache.nutch.storage.StorageUtils.createDataStore(StorageUtils.java:43)
          	at org.apache.nutch.storage.TestGoraStorage.setUp(TestGoraStorage.java:47)
          Caused by: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:747)
          	at org.apache.gora.sql.store.SqlStore.initialize(SqlStore.java:160)
          	at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:81)
          	at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:104)
          	... 14 more
          Caused by: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused
          	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
          	at org.hsqldb.jdbc.Util.sqlException(Unknown Source)
          	at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source)
          	at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source)
          	at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source)
          	at java.sql.DriverManager.getConnection(DriverManager.java:582)
          	at java.sql.DriverManager.getConnection(DriverManager.java:207)
          	at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:737)
          	... 17 more
          Caused by: org.hsqldb.HsqlException: java.net.ConnectException: Connection refused
          	at org.hsqldb.ClientConnection.openConnection(Unknown Source)
          	at org.hsqldb.ClientConnection.initConnection(Unknown Source)
          	at org.hsqldb.ClientConnection.<init>(Unknown Source)
          	... 23 more
          Caused by: java.net.ConnectException: Connection refused
          	at java.net.PlainSocketImpl.socketConnect(Native Method)
          	at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
          	at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
          	at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
          	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
          	at java.net.Socket.connect(Socket.java:529)
          	at java.net.Socket.connect(Socket.java:478)
          	at java.net.Socket.<init>(Socket.java:375)
          	at java.net.Socket.<init>(Socket.java:189)
          	at org.hsqldb.server.HsqlSocketFactory.createSocket(Unknown Source)
          	... 26 more
          
          FAILURES!!!
          Tests run: 2,  Failures: 0,  Errors: 2
          
          
          Show
          Lewis John McGibbney added a comment - full stack trace when attempting to run this test case lewis@lewis-01:~/ASF/nutchgora/runtime/local$ bin/nutch junit org.apache.nutch.storage.TestGoraStorage .E.E Time: 0.931 There were 2 errors: 1) testMultithread(org.apache.nutch.storage.TestGoraStorage)org.apache.gora.util.GoraException: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:110) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:87) at org.apache.nutch.storage.StorageUtils.createDataStore(StorageUtils.java:43) at org.apache.nutch.storage.TestGoraStorage.setUp(TestGoraStorage.java:47) Caused by: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:747) at org.apache.gora.sql.store.SqlStore.initialize(SqlStore.java:160) at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:81) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:104) ... 14 more Caused by: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source) at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source) at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:207) at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:737) ... 17 more Caused by: org.hsqldb.HsqlException: java.net.ConnectException: Connection refused at org.hsqldb.ClientConnection.openConnection(Unknown Source) at org.hsqldb.ClientConnection.initConnection(Unknown Source) at org.hsqldb.ClientConnection.<init>(Unknown Source) ... 23 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:529) at java.net.Socket.connect(Socket.java:478) at java.net.Socket.<init>(Socket.java:375) at java.net.Socket.<init>(Socket.java:189) at org.hsqldb.server.HsqlSocketFactory.createSocket(Unknown Source) ... 26 more 2) testMultiProcess(org.apache.nutch.storage.TestGoraStorage)org.apache.gora.util.GoraException: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:110) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:118) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:87) at org.apache.nutch.storage.StorageUtils.createDataStore(StorageUtils.java:43) at org.apache.nutch.storage.TestGoraStorage.setUp(TestGoraStorage.java:47) Caused by: java.io.IOException: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:747) at org.apache.gora.sql.store.SqlStore.initialize(SqlStore.java:160) at org.apache.gora.store.DataStoreFactory.initializeDataStore(DataStoreFactory.java:81) at org.apache.gora.store.DataStoreFactory.createDataStore(DataStoreFactory.java:104) ... 14 more Caused by: java.sql.SQLTransientConnectionException: java.net.ConnectException: Connection refused at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.Util.sqlException(Unknown Source) at org.hsqldb.jdbc.JDBCConnection.<init>(Unknown Source) at org.hsqldb.jdbc.JDBCDriver.getConnection(Unknown Source) at org.hsqldb.jdbc.JDBCDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:582) at java.sql.DriverManager.getConnection(DriverManager.java:207) at org.apache.gora.sql.store.SqlStore.getConnection(SqlStore.java:737) ... 17 more Caused by: org.hsqldb.HsqlException: java.net.ConnectException: Connection refused at org.hsqldb.ClientConnection.openConnection(Unknown Source) at org.hsqldb.ClientConnection.initConnection(Unknown Source) at org.hsqldb.ClientConnection.<init>(Unknown Source) ... 23 more Caused by: java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333) at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195) at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366) at java.net.Socket.connect(Socket.java:529) at java.net.Socket.connect(Socket.java:478) at java.net.Socket.<init>(Socket.java:375) at java.net.Socket.<init>(Socket.java:189) at org.hsqldb.server.HsqlSocketFactory.createSocket(Unknown Source) ... 26 more FAILURES!!! Tests run: 2, Failures: 0, Errors: 2

            People

            • Assignee:
              Lewis John McGibbney
              Reporter:
              Lewis John McGibbney
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development