Derby
  1. Derby
  2. DERBY-2872

Add Replication functionality to Derby

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.4.1.3
    • Fix Version/s: 10.4.1.3
    • Component/s: Replication
    • Labels:
      None

      Description

      It would be nice to have replication functionality to Derby; many potential Derby users seem to want this. The attached functional specification lists some initial thoughts for how this feature may work.

      Dag Wanvik had a look at this functionality some months ago. He wrote a proof of concept patch that enables replication by copying (using file system copy) and redoing the existing Derby transaction log to the slave (unfortunately, I can not find the mail thread now).

      DERBY-2852 contains a patch that enables replication by sending dedicated logical log records to the slave through a network connection and redoing these.

      Replication has been requested and discussed previously in multiple threads, including these:

      http://mail-archives.apache.org/mod_mbox/db-derby-user/200504.mbox/%3c426E04C1.1070904@yahoo.de%3e
      http://www.nabble.com/Does-Derby-support-Transaction-Logging---t2626667.html

      1. slave_classes_1.pdf
        4 kB
        Jørgen Løland
      2. ReplicationWriteup.txt
        11 kB
        V.Narayanan
      3. ReplicationDesign.pdf
        5 kB
        V.Narayanan
      4. replication_script.txt
        7 kB
        Jørgen Løland
      5. replication_funcspec.html
        6 kB
        Jørgen Løland
      6. replication_funcspec_v9.html
        29 kB
        Jørgen Løland
      7. replication_funcspec_v8.html
        26 kB
        Jørgen Løland
      8. replication_funcspec_v7.html
        23 kB
        Jørgen Løland
      9. replication_funcspec_v6.html
        17 kB
        Jørgen Løland
      10. replication_funcspec_v5.html
        12 kB
        Jørgen Løland
      11. replication_funcspec_v4.html
        13 kB
        V.Narayanan
      12. replication_funcspec_v3.html
        11 kB
        Jørgen Løland
      13. replication_funcspec_v2.html
        8 kB
        Jørgen Løland
      14. replication_funcspec_v10.html
        30 kB
        Jørgen Løland
      15. replication_funcspec_v10.html
        30 kB
        Jørgen Løland
      16. proof-of-concept_v2b-howto.txt
        4 kB
        Jørgen Løland
      17. proof_of_concept_slave.stat
        0.6 kB
        Jørgen Løland
      18. proof_of_concept_slave.diff
        18 kB
        Jørgen Løland
      19. proof_of_concept_master.stat
        0.4 kB
        Jørgen Løland
      20. proof_of_concept_master.diff
        65 kB
        Jørgen Løland
      21. poc_slave_v2c.diff
        56 kB
        V.Narayanan
      22. poc_slave_v2b.diff
        54 kB
        Jørgen Løland
      23. poc_slave_v2.stat
        0.7 kB
        Jørgen Løland
      24. poc_slave_v2.diff
        40 kB
        Jørgen Løland
      25. poc_master_v2b.diff
        44 kB
        Jørgen Løland
      26. poc_master_v2.stat
        0.7 kB
        Jørgen Løland
      27. poc_master_v2.diff
        35 kB
        Jørgen Løland
      28. master_classes_1.pdf
        5 kB
        Jørgen Løland

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Jørgen Løland created issue -
          Jørgen Løland made changes -
          Field Original Value New Value
          Attachment replication_funcspec.html [ 12360559 ]
          Jørgen Løland made changes -
          Link This issue is related to DERBY-2852 [ DERBY-2852 ]
          Jørgen Løland made changes -
          Attachment proof_of_concept_master.stat [ 12360915 ]
          Attachment proof_of_concept_master.diff [ 12360914 ]
          Jørgen Løland made changes -
          Attachment proof_of_concept_slave.diff [ 12360916 ]
          Attachment proof_of_concept_slave.stat [ 12360917 ]
          Attachment replication_script.txt [ 12360918 ]
          Jørgen Løland made changes -
          Assignee Jørgen Løland [ jorgenlo ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v2.html [ 12361293 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v2.html [ 12361294 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v2.html [ 12361293 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v3.html [ 12361474 ]
          Jørgen Løland made changes -
          Link This issue incorporates DERBY-2922 [ DERBY-2922 ]
          Jørgen Løland made changes -
          Link This issue incorporates DERBY-2978 [ DERBY-2978 ]
          V.Narayanan made changes -
          Attachment replication_funcspec_v4.html [ 12364057 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v5.html [ 12365078 ]
          Jørgen Løland made changes -
          Attachment master_classes_1.pdf [ 12365556 ]
          Jørgen Løland made changes -
          Attachment slave_classes_1.pdf [ 12365560 ]
          Jørgen Løland made changes -
          Attachment poc_master_v2.diff [ 12366445 ]
          Attachment poc_master_v2.stat [ 12366444 ]
          Jørgen Løland made changes -
          Attachment poc_slave_v2.stat [ 12366446 ]
          Attachment poc_slave_v2.diff [ 12366447 ]
          Jørgen Løland made changes -
          Attachment poc_slave_v2b.diff [ 12366733 ]
          Attachment proof-of-concept_v2b-howto.txt [ 12366734 ]
          Attachment poc_master_v2b.diff [ 12366732 ]
          V.Narayanan made changes -
          Attachment poc_slave_v2c.diff [ 12367463 ]
          Ole Solberg made changes -
          Link This issue blocks DERBY-3161 [ DERBY-3161 ]
          Jørgen Løland made changes -
          Link This issue incorporates DERBY-3169 [ DERBY-3169 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v6.html [ 12368813 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v7.html [ 12369351 ]
          Jørgen Løland made changes -
          Component/s Replication [ 12312050 ]
          Component/s Miscellaneous [ 11400 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v8.html [ 12372080 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v9.html [ 12375828 ]
          Henri van de Scheur made changes -
          Comment [ I have a few comments/questions to the Functional Specification for Derby Replication - rev. 9.0.

          Great to see a chapter 'Handling Failure Scenarios', but as stated here, there are numerous failure scenarios which can be encountered. As a follow-up, I would like to raise the following questions/comments:
          1. Would be nice to also describe a scenario where the slave Derby instance crashes
          2. Please confirm the following statement: 'Since the slave database is in recovery-mode, no data in the slave database can be changed at all by any means'. If this wouldn't be the case, a lot of consistency-problems and scenarios should be taken into account.
          3. Is the relation Master-Slave really an N:M-relation and if so, are N and/or M somehow limited?
          4. Master has still connection with Slave, but for some reason the Slave does not process the received logrecords: will this be seen as 'unexpected failure'?
          5. Network issue: what to do in case of missing packages?
          6. Network issue: will a multipath-configuration be supported? This will lead to a Slave receiving records in a wrong sequence
          7. Network issue: multiple-network-interfaces: question as for 6). In addition: special scenario(s) if one such interface is dropped?
          8. If 1 Master replicates to 2 different Slaves and one of these Slaves has a lower 'bandwidth' to receive/process the logs, will this one then determine the total (replication)transactional processes, thus also limit sending to the other Slave?
          9. If Derby instance I1 contains database D1 as Master, database D2 as Slave, instance I2 contains the same but with opposite roles: isn't there a big risk for a 'dead' race in case of problems with main memory log buffer?

          Thanks! ]
          V.Narayanan made changes -
          Attachment ReplicationWriteup.txt [ 12376937 ]
          Attachment ReplicationDesign.pdf [ 12376938 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v10.html [ 12377514 ]
          Jørgen Løland made changes -
          Attachment replication_funcspec_v10.html [ 12377517 ]
          Jørgen Løland made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Jørgen Løland made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Dag H. Wanvik made changes -
          Issue Type New Feature [ 2 ] Improvement [ 4 ]
          Kathey Marsden made changes -
          Fix Version/s 10.4.1.3 [ 12313111 ]
          Gavin made changes -
          Link This issue blocks DERBY-3161 [ DERBY-3161 ]
          Gavin made changes -
          Link This issue is depended upon by DERBY-3161 [ DERBY-3161 ]
          Gavin made changes -
          Workflow jira [ 12407061 ] Default workflow, editable Closed status [ 12802294 ]

            People

            • Assignee:
              Jørgen Løland
              Reporter:
              Jørgen Løland
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development