Derby
  1. Derby
  2. DERBY-3161 Test the Derby Replication functionality introduced in DERBY-2872
  3. DERBY-4417

Enable distributed replication testing - master server, slave server and test client running on different hosts.

    Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 10.6.1.0
    • Fix Version/s: None
    • Component/s: Replication, Test
    • Labels:
      None
    1. DERBY_distr_repl_A.diff.txt
      6 kB
      Ole Solberg
    2. DERBY_distr_repl_B.diff.txt
      41 kB
      Ole Solberg
    3. DERBY_distr_repl_C-replicationtest.properties
      4 kB
      Ole Solberg
    4. DERBY_distr_repl_D-start_script.txt
      2 kB
      Ole Solberg
    5. DERBY-4417_P1_p1.diff
      38 kB
      Ole Solberg
    6. DERBY-4417_P2_p1.diff.txt
      27 kB
      Ole Solberg

      Activity

      Hide
      Ole Solberg added a comment -

      Committed DERBY-4417_P2_p1.diff.txt to trunk as revision 895582.

      Show
      Ole Solberg added a comment - Committed DERBY-4417 _P2_p1.diff.txt to trunk as revision 895582.
      Hide
      Ole Solberg added a comment -

      DERBY-4417_P2_p1.diff.txt:
      ---------------------------
      This patch enables the ReplicationRun_Distributed test, reading a replication configuration from
      the replicationtest.properties file.

      1. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun_Distributed.java

      • Write the "dummy" ij script 'ij_dummy_script.sql'.
      • Read replication configuration from '$ {user.dir}

        /replicationtest.properties'.

      • Replace System.out.println() with Utils.DEBUG()

      2. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/SimplePerfTest.java

      • Enable specifying database path name for test client connection.

      3. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/SimplePerfTest_Verify.java
      Added. A simple verificationclient for SimplePerfTest.java just asserting correct number of rows in
      the test database (slave or master).

      4. java/testing/org/apache/derbyTesting/junit/TestConfiguration.java

      • added existingServerSuite(..., dbPath).Ditto for clientExistingServerSuite(), existingServerDecorator()
        and TestConfiguration().

      5. java/testing/org/apache/derbyTesting/junit/ServerSetup.java

      • added getNewConfiguration(..., dbPath) and ditto for ServerSetup().

      Depends on DERBY-4417_P1_p1.diff.

      Ran OK on suites.All and org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Distributed.

      ReplicationRun_Distributed is not wired into suites.All. It is intended to be run "manually".

      Show
      Ole Solberg added a comment - DERBY-4417 _P2_p1.diff.txt: --------------------------- This patch enables the ReplicationRun_Distributed test, reading a replication configuration from the replicationtest.properties file. 1. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun_Distributed.java Write the "dummy" ij script 'ij_dummy_script.sql'. Read replication configuration from '$ {user.dir} /replicationtest.properties'. Replace System.out.println() with Utils.DEBUG() 2. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/SimplePerfTest.java Enable specifying database path name for test client connection. 3. java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/SimplePerfTest_Verify.java Added. A simple verificationclient for SimplePerfTest.java just asserting correct number of rows in the test database (slave or master). 4. java/testing/org/apache/derbyTesting/junit/TestConfiguration.java added existingServerSuite(..., dbPath).Ditto for clientExistingServerSuite(), existingServerDecorator() and TestConfiguration(). 5. java/testing/org/apache/derbyTesting/junit/ServerSetup.java added getNewConfiguration(..., dbPath) and ditto for ServerSetup(). Depends on DERBY-4417 _P1_p1.diff. Ran OK on suites.All and org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Distributed. ReplicationRun_Distributed is not wired into suites.All. It is intended to be run "manually".
      Hide
      Ole Solberg added a comment -

      Committed DERBY-4417_P1_p1.diff to trunk as revision 894607.

      Show
      Ole Solberg added a comment - Committed DERBY-4417 _P1_p1.diff to trunk as revision 894607.
      Hide
      Ole Solberg added a comment -

      DERBY-4417_P1_p1.diff.txt intended for commit.

      Show
      Ole Solberg added a comment - DERBY-4417 _P1_p1.diff.txt intended for commit.
      Hide
      Ole Solberg added a comment -

      DERBY-4417_P1_p1.diff.txt:
      --------------------------
      This patch is a preparation for patches to enable distributed replication testing.
      The patch touches only java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun.java:

      1. Allow a plain class w/main(), i.e. NOT using JUnit, as a client load when running distributed.

      2. Introduced THREADS, MINUTES in preparation for the ReplicationTestRunStress/StressMultiTest load client.

      3. Removed using pre-installed dummy ij script - still creates the script in $

      {HOME}

      .

      . Various cleanups:

      • Removing unused code.
      • Introduced ReplicationRun.get {Master|Slave}

        JavaExecutableName() to handle
        the equivalent of BaseTestCase.getJavaExecutableName() also when running
        distributed: allowing jvms to be specified in replicationtest.properties.

      org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite and
      org.apache.derbyTesting.functionTests.suites.All ran OK.
      org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Distributed
      also ran OK with various loads.

      Planning to commit later today.

      Show
      Ole Solberg added a comment - DERBY-4417 _P1_p1.diff.txt: -------------------------- This patch is a preparation for patches to enable distributed replication testing. The patch touches only java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun.java: 1. Allow a plain class w/main(), i.e. NOT using JUnit, as a client load when running distributed. 2. Introduced THREADS, MINUTES in preparation for the ReplicationTestRunStress/StressMultiTest load client. 3. Removed using pre-installed dummy ij script - still creates the script in $ {HOME} . . Various cleanups: Removing unused code. Introduced ReplicationRun.get {Master|Slave} JavaExecutableName() to handle the equivalent of BaseTestCase.getJavaExecutableName() also when running distributed: allowing jvms to be specified in replicationtest.properties. org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite and org.apache.derbyTesting.functionTests.suites.All ran OK. org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Distributed also ran OK with various loads. Planning to commit later today.
      Hide
      Ole Solberg added a comment -

      Thanks, Dag!
      I will fix these in the next upload of the patch - the printlns are from my testing, so are the name/"param" mismatch.
      It's a good idea to make those numbers parameterizable. I agree the test should be renamed.

      Show
      Ole Solberg added a comment - Thanks, Dag! I will fix these in the next upload of the patch - the printlns are from my testing, so are the name/"param" mismatch. It's a good idea to make those numbers parameterizable. I agree the test should be renamed.
      Hide
      Dag H. Wanvik added a comment -

      Thanks, Ole!
      I know you said these are preliminary, so my comments may be premature.
      Just browsing through I noticed that you use System.out.println; is there a reason why you don't use
      the println that's available (inherited) from BaseTestCase (prints conditionally if verbose set, i.e. derby.tests.debug).
      I also notice that StressMultiClient50x59 is misnamed; it actually uses 100 by 30, should these numbers be parameterizable? And/or rename test? I this the name XStressMultiTest a could be more descriptive?

      Show
      Dag H. Wanvik added a comment - Thanks, Ole! I know you said these are preliminary, so my comments may be premature. Just browsing through I noticed that you use System.out.println; is there a reason why you don't use the println that's available (inherited) from BaseTestCase (prints conditionally if verbose set, i.e. derby.tests.debug). I also notice that StressMultiClient50x59 is misnamed; it actually uses 100 by 30, should these numbers be parameterizable? And/or rename test? I this the name XStressMultiTest a could be more descriptive?
      Hide
      Ole Solberg added a comment -

      Attaching preliminary patches to run distributed replication testing - i.e. master server, slave server and test client running on different hosts.

      DERBY_distr_repl_A.diff.txt
      -------------------------------------
      M java/testing/org/apache/derbyTesting/functionTests/tests/multi/StressMultiTest.java

      Factoring out decorator code to make it available for cases when running against an
      already existing server.
      Added clientSuite() to experiment with different number of threads and running time
      on client/server.

      DERBY_distr_repl_B.diff.txt
      -------------------------------------
      A java/testing/org/apache/derbyTesting/functionTests/tests/multi/XStressMultiTest.java
      Run StressMultiTest against an existing server.

      A java/testing/org/apache/derbyTesting/functionTests/tests/multi/StressMultiClient50x59.java
      Experimenting with various settings for standard (localhost) StressMultiTest. Using
      clientServer() added in StressMultiTest().

      M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/StandardTests.java
      Added stressMultiTest() that sets up the XStressMultiTest suite.

      M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationTestRun.java
      Set to run without SecurityManager.

      M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun.java
      Enable to use jvms specified in replicationtest.properties file.
      NB. This needs further work to combine original use of BaseTestCase.getJavaExecutableName()
      and jvms specified in replicationtest.properties file.

      A java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationTestRunStress.java
      Replication test load using StandardTest.stressMultiTest = XStressMultiTest.

      M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun_Full.java
      Added initEnvironment() to get replication test configuration parameters from
      replicationtest.properties file located in "user.dir".

      DERBY_distr_repl_C-replicationtest.properties
      ---------------------------------------------------------------
      Example replicationtest.properties to be copied to "user.dir" before starting distributed replication tests.

      DERBY_distr_repl_D-start_script.txt
      ------------------------------------------------
      Example script to set up and run a distributed replication test.

      Show
      Ole Solberg added a comment - Attaching preliminary patches to run distributed replication testing - i.e. master server, slave server and test client running on different hosts. DERBY_distr_repl_A.diff.txt ------------------------------------- M java/testing/org/apache/derbyTesting/functionTests/tests/multi/StressMultiTest.java Factoring out decorator code to make it available for cases when running against an already existing server. Added clientSuite() to experiment with different number of threads and running time on client/server. DERBY_distr_repl_B.diff.txt ------------------------------------- A java/testing/org/apache/derbyTesting/functionTests/tests/multi/XStressMultiTest.java Run StressMultiTest against an existing server. A java/testing/org/apache/derbyTesting/functionTests/tests/multi/StressMultiClient50x59.java Experimenting with various settings for standard (localhost) StressMultiTest. Using clientServer() added in StressMultiTest(). M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/StandardTests.java Added stressMultiTest() that sets up the XStressMultiTest suite. M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationTestRun.java Set to run without SecurityManager. M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun.java Enable to use jvms specified in replicationtest.properties file. NB. This needs further work to combine original use of BaseTestCase.getJavaExecutableName() and jvms specified in replicationtest.properties file. A java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationTestRunStress.java Replication test load using StandardTest.stressMultiTest = XStressMultiTest. M java/testing/org/apache/derbyTesting/functionTests/tests/replicationTests/ReplicationRun_Full.java Added initEnvironment() to get replication test configuration parameters from replicationtest.properties file located in "user.dir". DERBY_distr_repl_C-replicationtest.properties --------------------------------------------------------------- Example replicationtest.properties to be copied to "user.dir" before starting distributed replication tests. DERBY_distr_repl_D-start_script.txt ------------------------------------------------ Example script to set up and run a distributed replication test.

        People

        • Assignee:
          Ole Solberg
          Reporter:
          Ole Solberg
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:

            Development