Derby
  1. Derby
  2. DERBY-2922 Replication: Add master replication mode
  3. DERBY-2977

Replication: Add a ReplicationMaster controller that will manage replication on the master side

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.4.1.3
    • Fix Version/s: 10.4.1.3
    • Component/s: Replication
    • Labels:
      None

      Description

      The replication master role includes many tasks:

      • set up a network connection to the slave
      • sending the database to the slave before starting replication
      • make sure that log records are appended to the network buffer, and that the log is later sent to the slave
      • etc

      This issue is for adding a controller that will start/stop/initiate all services needed for the replication master role.

      1. repli_logbuffer_v2d.diff
        44 kB
        Jørgen Løland
      2. repli_logbuffer_v2d.stat
        0.7 kB
        Jørgen Løland
      3. repli_logbuffer_v2c.stat
        0.7 kB
        Jørgen Løland
      4. repli_logbuffer_v2c.diff
        44 kB
        Jørgen Løland
      5. repli_logbuffer_v2b.diff
        41 kB
        Jørgen Løland
      6. repli_logbuffer_v2b.stat
        0.6 kB
        Jørgen Løland
      7. repli_logbuffer_v2.diff
        42 kB
        Jørgen Løland
      8. repli_logbuffer_v2.stat
        0.6 kB
        Jørgen Løland
      9. derby_2977_1c.diff
        27 kB
        Jørgen Løland
      10. derby_2977_1c.stat
        0.9 kB
        Jørgen Løland
      11. derby_2977_1b.diff
        29 kB
        Jørgen Løland
      12. derby_2977_1b.stat
        1 kB
        Jørgen Løland
      13. derby_2977_1.diff
        27 kB
        Jørgen Løland
      14. derby_2977_1.stat
        1 kB
        Jørgen Løland

        Issue Links

          Activity

          Hide
          Jørgen Løland added a comment -

          Only remaining task is to ship the database from the master to the slave location using the replication network at replication startup. Filed DERBY-3434 for this.

          Show
          Jørgen Løland added a comment - Only remaining task is to ship the database from the master to the slave location using the replication network at replication startup. Filed DERBY-3434 for this.
          Hide
          Øystein Grøvlen added a comment -

          Committed patch, repli_logbuffer_v2d.diff, as revision 585900.

          Show
          Øystein Grøvlen added a comment - Committed patch, repli_logbuffer_v2d.diff, as revision 585900.
          Hide
          Jørgen Løland added a comment -

          Thanks for reviewing patch v2c Øystein.

          In patch v2d, I left the implementation of your 2nd comment as is since you don't seem to feel very strongly about it. The other issues have been fixed. All tests passed.

          Show
          Jørgen Løland added a comment - Thanks for reviewing patch v2c Øystein. In patch v2d, I left the implementation of your 2nd comment as is since you don't seem to feel very strongly about it. The other issues have been fixed. All tests passed.
          Hide
          Øystein Grøvlen added a comment -

          Thanks for addressing my comments in the new patch, Jørgen. I have
          just a few minor follow-up comments (same numbering as in my previous
          review):

          1. Your basically looks good. IMHO, calling the new method
          'checkForReplication' or something instead of
          'initLogAccessFileReplication' since it will not always init
          replication. (Check is also the main verb you use to describe in
          the javadoc what it is doing.)

          As an alternative to 'startReplicationMasterRole', I suggest
          'setReplicationMasterRole', since as I said, it is not always
          starting replication.

          2. A third alternative implementation that will behave almost the same
          as the current strategy as seen from the user, is to remember any
          logFileSwitch at the end of a log chunk until the next chunk
          arrives, and let switchLogFile() be true for the first log record
          of the next chunk. Then, removing 'else' will be the only necessary change
          to your example code.

          4. As far as I can see, you have not addressed this typo.

          9. I think I meant LogAccessFile#stopReplicationMasterRole. Anyway,
          it is fixed now.

          10. (NEW) To me, adding instant as a parameter to
          LogAccessFile#writeToLog seems like a better solution than
          maintaining a field 'instantCurrentChunk'.

          None of the above issues are very important, and it is your
          call whether you would like to fix any of them before I commit.

          Show
          Øystein Grøvlen added a comment - Thanks for addressing my comments in the new patch, Jørgen. I have just a few minor follow-up comments (same numbering as in my previous review): 1. Your basically looks good. IMHO, calling the new method 'checkForReplication' or something instead of 'initLogAccessFileReplication' since it will not always init replication. (Check is also the main verb you use to describe in the javadoc what it is doing.) As an alternative to 'startReplicationMasterRole', I suggest 'setReplicationMasterRole', since as I said, it is not always starting replication. 2. A third alternative implementation that will behave almost the same as the current strategy as seen from the user, is to remember any logFileSwitch at the end of a log chunk until the next chunk arrives, and let switchLogFile() be true for the first log record of the next chunk. Then, removing 'else' will be the only necessary change to your example code. 4. As far as I can see, you have not addressed this typo. 9. I think I meant LogAccessFile#stopReplicationMasterRole. Anyway, it is fixed now. 10. (NEW) To me, adding instant as a parameter to LogAccessFile#writeToLog seems like a better solution than maintaining a field 'instantCurrentChunk'. None of the above issues are very important, and it is your call whether you would like to fix any of them before I commit.
          Hide
          Jørgen Løland added a comment -

          Patch v2c is a minor modification to v2b (added some javadoc and a simple log instant change for log chunks in LogAccessFile). It replaces v2b. All tests passed

          Show
          Jørgen Løland added a comment - Patch v2c is a minor modification to v2b (added some javadoc and a simple log instant change for log chunks in LogAccessFile). It replaces v2b. All tests passed
          Hide
          Jørgen Løland added a comment -

          Thanks for the review, Øystein. I have attached a new patch, v2b, addressing all your comments except 2:

          1. Changed constructor of LogAccessFile to call LogToFile#initLogAccessFileReplication. I cannot think of a better name than startReplicationMasterRole, but feel free to suggest one...
          9. Do you mean LogToFile#start/stop? If so, startReplicationMasterRole uses this.masterFactory because a parameter has the same name, which is not the case for stopReplicationMasterRole.

          Not fixed:

          2. To be honest, I don't like either of these strategies

          With the current strategy:
          next() returns true -> some info is available - could be a log file switch, could be a log record
          next() returns false -> done

          This allows the loop:
          while (next())

          { if (scan.isFileSwitch()) switchLogFile(); else applyLog(scan.getData()); }

          The suggested strategy:
          next() returns true -> log available, maybe also a log file switch
          next() returns false -> no log available, but maybe a log file switch?

          I'm ok with the "true" scenario, but not the "false" scenario. I.e., I don't think it is ok for ReplicationLogScan to have vital information (a log switch must be performed at the same place on the slave and master) when it returns false. I did not change this in the attached patch. Ideally, I would like to see a third strategy that was better than both these, but cannot think of
          one.

          All tests pass cleanly.

          Show
          Jørgen Løland added a comment - Thanks for the review, Øystein. I have attached a new patch, v2b, addressing all your comments except 2: 1. Changed constructor of LogAccessFile to call LogToFile#initLogAccessFileReplication. I cannot think of a better name than startReplicationMasterRole, but feel free to suggest one... 9. Do you mean LogToFile#start/stop? If so, startReplicationMasterRole uses this.masterFactory because a parameter has the same name, which is not the case for stopReplicationMasterRole. Not fixed: 2. To be honest, I don't like either of these strategies With the current strategy: next() returns true -> some info is available - could be a log file switch, could be a log record next() returns false -> done This allows the loop: while (next()) { if (scan.isFileSwitch()) switchLogFile(); else applyLog(scan.getData()); } The suggested strategy: next() returns true -> log available, maybe also a log file switch next() returns false -> no log available, but maybe a log file switch? I'm ok with the "true" scenario, but not the "false" scenario. I.e., I don't think it is ok for ReplicationLogScan to have vital information (a log switch must be performed at the same place on the slave and master) when it returns false. I did not change this in the attached patch. Ideally, I would like to see a third strategy that was better than both these, but cannot think of one. All tests pass cleanly.
          Hide
          Øystein Grøvlen added a comment -

          The patch looks very good. I have a few comments, but most are minor nits:

          1. LogToFile:
          It seems a bit strange to add checks for replicationMasterMode in
          situations where replication should not be going on (e.g., during
          recovery). Would it be better to add a method to LogToFile that
          the LogAccessFile constructor could use to check whether
          replication was going on? Or maybe pass it as a parameter to the
          constructor? With the current usage, I would change the name of
          LogAccessFile#startReplicationMasterRole since it is not called
          only when replication is started.

          2. ReplicationLogScan:
          I am not quite sure I like the way you handle the mix of log
          records and log file switch records. I wonder whether it would be
          cleaner if when a log file switch is encountered, one would set the
          logFileSwitch flag and continue to read the next log record. Then,
          if next() returns true there will always be a valid log record, but
          you will have to check for log file switch before you handle it.

          3. ReplicationLogScan, javadoc:
          Why does the @see list the same class twice?

          4. ReplicationLogScan#next, comment in exception handling (not new):
          Typo: (currupt)

          5. ReplicationLogBuffer, javadoc (Not added by this patch):
          I feel "... consists of n LogBufferElement, each of which can store
          m log records ..." is a bit confusing since it sounds like the
          number of log records in a buffer (m) is a configurable constant.

          6. ReplicationLogBuffer#appendBytes:
          You do not want javadoc for the params?

          7. MasterController:
          Import of ReplicationMessage is unused.

          8. MasterController/MasterFactory#appendLogRecord, javadoc:
          Mismatch between javadoc and parameter name for length.

          9. MasterController#startReplicationMasterRole/stopReplicationMasterRole:
          Why is one of the member fields prefixed with 'this.', but not the
          other?

          Show
          Øystein Grøvlen added a comment - The patch looks very good. I have a few comments, but most are minor nits: 1. LogToFile: It seems a bit strange to add checks for replicationMasterMode in situations where replication should not be going on (e.g., during recovery). Would it be better to add a method to LogToFile that the LogAccessFile constructor could use to check whether replication was going on? Or maybe pass it as a parameter to the constructor? With the current usage, I would change the name of LogAccessFile#startReplicationMasterRole since it is not called only when replication is started. 2. ReplicationLogScan: I am not quite sure I like the way you handle the mix of log records and log file switch records. I wonder whether it would be cleaner if when a log file switch is encountered, one would set the logFileSwitch flag and continue to read the next log record. Then, if next() returns true there will always be a valid log record, but you will have to check for log file switch before you handle it. 3. ReplicationLogScan, javadoc: Why does the @see list the same class twice? 4. ReplicationLogScan#next, comment in exception handling (not new): Typo: (currupt) 5. ReplicationLogBuffer, javadoc (Not added by this patch): I feel "... consists of n LogBufferElement, each of which can store m log records ..." is a bit confusing since it sounds like the number of log records in a buffer (m) is a configurable constant. 6. ReplicationLogBuffer#appendBytes: You do not want javadoc for the params? 7. MasterController: Import of ReplicationMessage is unused. 8. MasterController/MasterFactory#appendLogRecord, javadoc: Mismatch between javadoc and parameter name for length. 9. MasterController#startReplicationMasterRole/stopReplicationMasterRole: Why is one of the member fields prefixed with 'this.', but not the other?
          Hide
          Jørgen Løland added a comment -

          See DERBY-2872 for version 2 of the proof of concept.

          Show
          Jørgen Løland added a comment - See DERBY-2872 for version 2 of the proof of concept.
          Hide
          Jørgen Løland added a comment -

          Version 2 of the proof of concept code showed a problem with the way log is appended to the repliation buffer. Currently, LogToFile#appendLogRecord is used to write log to the replication master log buffer. However, the LogAccessFile class that wraps write operations to log file also write checksums. We need these checksums at the slave because without them, the log instants on the slave and master will differ, in turn making recovery impossible (undo operations refer to a specific instant, which will not be correct).

          The attached patch, repli_logbuffer_v2.*, changes this erroneous behavior. With this patch, log is appended to the repliation log buffer in LogAccessFile, not LogToFile like now. The patch modifies the following files:

          M java/engine/org/apache/derby/impl/services/replication/buffer/LogBufferElement.java
          M java/engine/org/apache/derby/impl/services/replication/buffer/ReplicationLogBuffer.java

          The buffer now accepts chunks of log records from LogAccessFile#writeToLog (i.e., appended to replication buffer in the same method that writes the log to disk) instead of single log records from LogToFile#appendLogRecord

          M java/engine/org/apache/derby/impl/services/replication/slave/ReplicationLogScan.java

          Modified the slave-side log parser to read the new chunk of log record format

          M java/engine/org/apache/derby/impl/services/replication/master/MasterController.java
          M java/engine/org/apache/derby/iapi/services/replication/master/MasterFactory.java

          Modified appendLog signature to accept log from LogAccessFile instead of from LogToFile

          M java/engine/org/apache/derby/impl/store/raw/log/LogAccessFile.java

          Can be set inReplicationMasterMode, in which log is appended to MasterFactory

          M java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java

          Removed the code that appends log to replication log buffer, and tells LogAccessFile to go into replication master mode when needed.

          Allsuites and derbyall completed without error

          Show
          Jørgen Løland added a comment - Version 2 of the proof of concept code showed a problem with the way log is appended to the repliation buffer. Currently, LogToFile#appendLogRecord is used to write log to the replication master log buffer. However, the LogAccessFile class that wraps write operations to log file also write checksums. We need these checksums at the slave because without them, the log instants on the slave and master will differ, in turn making recovery impossible (undo operations refer to a specific instant, which will not be correct). The attached patch, repli_logbuffer_v2.*, changes this erroneous behavior. With this patch, log is appended to the repliation log buffer in LogAccessFile, not LogToFile like now. The patch modifies the following files: M java/engine/org/apache/derby/impl/services/replication/buffer/LogBufferElement.java M java/engine/org/apache/derby/impl/services/replication/buffer/ReplicationLogBuffer.java The buffer now accepts chunks of log records from LogAccessFile#writeToLog (i.e., appended to replication buffer in the same method that writes the log to disk) instead of single log records from LogToFile#appendLogRecord M java/engine/org/apache/derby/impl/services/replication/slave/ReplicationLogScan.java Modified the slave-side log parser to read the new chunk of log record format M java/engine/org/apache/derby/impl/services/replication/master/MasterController.java M java/engine/org/apache/derby/iapi/services/replication/master/MasterFactory.java Modified appendLog signature to accept log from LogAccessFile instead of from LogToFile M java/engine/org/apache/derby/impl/store/raw/log/LogAccessFile.java Can be set inReplicationMasterMode, in which log is appended to MasterFactory M java/engine/org/apache/derby/impl/store/raw/log/LogToFile.java Removed the code that appends log to replication log buffer, and tells LogAccessFile to go into replication master mode when needed. Allsuites and derbyall completed without error
          Hide
          Øystein Grøvlen added a comment -

          Committed patch derby_2977_1c.stat.

          Show
          Øystein Grøvlen added a comment - Committed patch derby_2977_1c.stat.
          Hide
          Øystein Grøvlen added a comment -

          Thanks Jørgen. What you say makes sense. I have started the tests, and I will commit the latest patch tomorrow if the tests run well.

          Show
          Øystein Grøvlen added a comment - Thanks Jørgen. What you say makes sense. I have started the tests, and I will commit the latest patch tomorrow if the tests run well.
          Hide
          Jørgen Løland added a comment -

          Thanks for the review of the patch, Øystein.

          Attaching a patch, v1c, addressing the buffer size issue. I agree that the replication buffer size should not depend on the disk buffer size. In lack of a better size, I still use 32k, but now defined in a constant in MasterController. This size will probably be subject to changes once we are able to test replication performance

          Regarding flushedTo: I do not see any good reason why we should not allow log records to be sent to the slave before these are written to disk. Any operations that are sent prematurely to the slave will be aborted if a commit has not been received on the slave. The disk buffer also works this way - it may write log to disk even when force has not been called.

          There is, however, a possibility that the following sequence happens during commit processing on the master:

          1. Commit log record is appended to disk buffer
          2. Commit log record is appended to replication buffer
          3. Replication log shipping thread gets CPU time and ships all buffered log to the slave
          4. Master crashes before commit log record has been safely written to disk.

          This would lead to a case where the commit log record is not on disk on the master, but is on the slave. However, I do not think this is a problem because:

          • Commit has been requested in the first place, so the client should agree that committing was the intention.
          • For the operations of that transaction to be seen by anyone connecting to the previous slave, the failover command must be issued on the slave. The functional specification states that when this happens, the database that had the master role is no longer the active database. Thus, the actual disk image of the crashed master database is not interesting.

          I have started the tests, but expect no failures since the code in the patch is not in use. I will revert with the results as soon as the tests complete.

          Show
          Jørgen Løland added a comment - Thanks for the review of the patch, Øystein. Attaching a patch, v1c, addressing the buffer size issue. I agree that the replication buffer size should not depend on the disk buffer size. In lack of a better size, I still use 32k, but now defined in a constant in MasterController. This size will probably be subject to changes once we are able to test replication performance Regarding flushedTo: I do not see any good reason why we should not allow log records to be sent to the slave before these are written to disk. Any operations that are sent prematurely to the slave will be aborted if a commit has not been received on the slave. The disk buffer also works this way - it may write log to disk even when force has not been called. There is, however, a possibility that the following sequence happens during commit processing on the master: 1. Commit log record is appended to disk buffer 2. Commit log record is appended to replication buffer 3. Replication log shipping thread gets CPU time and ships all buffered log to the slave 4. Master crashes before commit log record has been safely written to disk. This would lead to a case where the commit log record is not on disk on the master, but is on the slave. However, I do not think this is a problem because: Commit has been requested in the first place, so the client should agree that committing was the intention. For the operations of that transaction to be seen by anyone connecting to the previous slave, the failover command must be issued on the slave. The functional specification states that when this happens, the database that had the master role is no longer the active database. Thus, the actual disk image of the crashed master database is not interesting. I have started the tests, but expect no failures since the code in the patch is not in use. I will revert with the results as soon as the tests complete.
          Hide
          Øystein Grøvlen added a comment -

          Jørgen Løland (JIRA) wrote:
          > Thanks for reviewing the patch, Øystein.
          >
          > The attached patch, v1b, addresses your comments in the following way:
          >
          >> 1. Why do you need to expose LogToFile#logBufferSize outside the
          >> class? The Logger used by transactions handles the buffer size
          >> dynamically, why cannot this be done in this case, too?
          >
          > Actually, as far as I can tell, the log buffer used by
          > transactions (LogAccessFile) uses the same LogToFile#buffersize
          > exposed here. Is there another file I should look at? I use this
          > buffer size because I cannot think of any other size that would
          > be better in the general case. I can change this if you have any
          > suggestions for an appropriate size. If so, MasterFactory should
          > probably store the default size as a static final variable. Left
          > unchanged in v1b since I can't think of a better alternative.

          Sorry, I thought there was one buffer per log record, which is similar
          to FileLogger which has a dynamic buffer size. Still, I do not think
          that what is the optimal buffer size for writing to disk is
          necessarily the best buffer size when sending messages over the
          network. I would think that should be independent of each other.

          The rest of the changes looks very good, but I have one question on
          flushedTo: It seems to be the assumption that a log record should not
          be sent to the slave before it is written to disk? Is this a
          necessary requirement? If yes, maybe that should be documented too.

          Show
          Øystein Grøvlen added a comment - Jørgen Løland (JIRA) wrote: > Thanks for reviewing the patch, Øystein. > > The attached patch, v1b, addresses your comments in the following way: > >> 1. Why do you need to expose LogToFile#logBufferSize outside the >> class? The Logger used by transactions handles the buffer size >> dynamically, why cannot this be done in this case, too? > > Actually, as far as I can tell, the log buffer used by > transactions (LogAccessFile) uses the same LogToFile#buffersize > exposed here. Is there another file I should look at? I use this > buffer size because I cannot think of any other size that would > be better in the general case. I can change this if you have any > suggestions for an appropriate size. If so, MasterFactory should > probably store the default size as a static final variable. Left > unchanged in v1b since I can't think of a better alternative. Sorry, I thought there was one buffer per log record, which is similar to FileLogger which has a dynamic buffer size. Still, I do not think that what is the optimal buffer size for writing to disk is necessarily the best buffer size when sending messages over the network. I would think that should be independent of each other. The rest of the changes looks very good, but I have one question on flushedTo: It seems to be the assumption that a log record should not be sent to the slave before it is written to disk? Is this a necessary requirement? If yes, maybe that should be documented too.
          Hide
          Jørgen Løland added a comment -

          Thanks for reviewing the patch, Øystein.

          The attached patch, v1b, addresses your comments in the following way:

          > 1. Why do you need to expose LogToFile#logBufferSize outside the
          > class? The Logger used by transactions handles the buffer size
          > dynamically, why cannot this be done in this case, too?

          Actually, as far as I can tell, the log buffer used by
          transactions (LogAccessFile) uses the same LogToFile#buffersize
          exposed here. Is there another file I should look at? I use this
          buffer size because I cannot think of any other size that would
          be better in the general case. I can change this if you have any
          suggestions for an appropriate size. If so, MasterFactory should
          probably store the default size as a static final variable. Left
          unchanged in v1b since I can't think of a better alternative.

          2-6: fixed in v1b.

          Show
          Jørgen Løland added a comment - Thanks for reviewing the patch, Øystein. The attached patch, v1b, addresses your comments in the following way: > 1. Why do you need to expose LogToFile#logBufferSize outside the > class? The Logger used by transactions handles the buffer size > dynamically, why cannot this be done in this case, too? Actually, as far as I can tell, the log buffer used by transactions (LogAccessFile) uses the same LogToFile#buffersize exposed here. Is there another file I should look at? I use this buffer size because I cannot think of any other size that would be better in the general case. I can change this if you have any suggestions for an appropriate size. If so, MasterFactory should probably store the default size as a static final variable. Left unchanged in v1b since I can't think of a better alternative. 2-6: fixed in v1b.
          Hide
          Øystein Grøvlen added a comment -

          I think this patch looks very good. Most of my comments are minor:

          1. Why do you need to expose LogToFile#logBufferSize outside the
          class? The Logger used by transactions handles the buffer size
          dynamically, why cannot this be done in this case, too?

          2. MasterFactory#flushedTo: The Javadoc should say something
          about why this method is needed and when it will be called.

          3. I think it would be good if it is were more clear from the
          MasterController code, which methods were implementation of which
          interface. E.g., have separate sections of the file for each of
          the interfaces with a heading in a comment.

          4. MasterController#canSupport: The javadoc for the return value
          should say something about what it means, not just its
          current implementation.

          5. ReadOnly: The new import does not seem necessary.

          6. MasterFactory javadoc says "This is an implementation ...", but it
          is not. It is the interface. MasterController is the
          implementation. I also think the note about being far from
          complete belongs in the class and not the interface. (There is
          also a missing "to" in the second sentence of the javadoc.)

          Show
          Øystein Grøvlen added a comment - I think this patch looks very good. Most of my comments are minor: 1. Why do you need to expose LogToFile#logBufferSize outside the class? The Logger used by transactions handles the buffer size dynamically, why cannot this be done in this case, too? 2. MasterFactory#flushedTo: The Javadoc should say something about why this method is needed and when it will be called. 3. I think it would be good if it is were more clear from the MasterController code, which methods were implementation of which interface. E.g., have separate sections of the file for each of the interfaces with a heading in a comment. 4. MasterController#canSupport: The javadoc for the return value should say something about what it means, not just its current implementation. 5. ReadOnly: The new import does not seem necessary. 6. MasterFactory javadoc says "This is an implementation ...", but it is not. It is the interface. MasterController is the implementation. I also think the note about being far from complete belongs in the class and not the interface. (There is also a missing "to" in the second sentence of the javadoc.)
          Hide
          Jørgen Løland added a comment -

          Attaching a patch, v1, intended to solve step 1 described above.

          The patch adds a service (MasterFactory (interface), MasterController) that will be used to manage replication on the master side. Currently, the service does very little work except write out a few "MasterController started" lines. These printouts will be removed when code is added to the service.

          The service is started by calling Database#startReplicationMaster on the database that will be replicated. However, there is no code calling this method yet, so the patch is currently not used for anything. The intention is that command line calls to NetworkServerControl will start the service. DERBY-2954 has been added for that task.

          Show
          Jørgen Løland added a comment - Attaching a patch, v1, intended to solve step 1 described above. The patch adds a service (MasterFactory (interface), MasterController) that will be used to manage replication on the master side. Currently, the service does very little work except write out a few "MasterController started" lines. These printouts will be removed when code is added to the service. The service is started by calling Database#startReplicationMaster on the database that will be replicated. However, there is no code calling this method yet, so the patch is currently not used for anything. The intention is that command line calls to NetworkServerControl will start the service. DERBY-2954 has been added for that task.
          Hide
          Jørgen Løland added a comment -

          I plan to implement the ReplicationMaster Controller incrementally in multiple steps:

          1) Add the basic code to Derby so that the master controller can be booted as a service when a "startmaster" command is issued to NetworkServerControl
          2-n) Incrementally add controller logic as replication patches are added to Derby.

          The current plan for step 1 is:

          • NetworkServerControl will lookup the correct database, and call a startReplicationMaster method on that database service. This call will propagate through the database and access layers to RawStore.java.
          • RawStore#startReplicationMaster will call Monitor#bootServiceModule on the MasterFactory (interface for MasterController)
          • When booted, the MasterController service will initialize some basic variables but do nothing fancy.

          Comments are appreciated.

          Show
          Jørgen Løland added a comment - I plan to implement the ReplicationMaster Controller incrementally in multiple steps: 1) Add the basic code to Derby so that the master controller can be booted as a service when a "startmaster" command is issued to NetworkServerControl 2-n) Incrementally add controller logic as replication patches are added to Derby. The current plan for step 1 is: NetworkServerControl will lookup the correct database, and call a startReplicationMaster method on that database service. This call will propagate through the database and access layers to RawStore.java. RawStore#startReplicationMaster will call Monitor#bootServiceModule on the MasterFactory (interface for MasterController) When booted, the MasterController service will initialize some basic variables but do nothing fancy. Comments are appreciated.
          Hide
          Jørgen Løland added a comment -

          The initial plan for the replication master controller:

          The replication master controller is started when the "start replication master" command (see functional specification in
          DERBY-2872) is issued for a database. There will be exactly one replication controller instance for every database with a master role.

          It seems to be a good idea to boot the replication controller as a service module on a per need basis. This is, e.g., done to boot the TransactionFactory and DataFactory services in engine/org/apache/derby/impl/store/raw/RawStore.java

          RawStore.java#boot, line 215-221:
          ----8<----
          xactFactory = (TransactionFactory)
          Monitor.bootServiceModule(
          create, this, getTransactionFactoryModule(), properties);

          dataFactory = (DataFactory)
          Monitor.bootServiceModule(
          create, this, getDataFactoryModule(), properties);
          ---->8----

          The module will need an API for these commands:

          start replication
          stop replication

          The controller will be managing the replication strategy (asynchronous, 1-safe, 2-safe, very-safe). For now, only
          asynchronous is considered. If more strategies are added later, the API should probably be extended with methods to dynamically change the replication strategy.

          Development on this issue will likely be incremental - the service must be modified as more functionality is in place.

          Show
          Jørgen Løland added a comment - The initial plan for the replication master controller: The replication master controller is started when the "start replication master" command (see functional specification in DERBY-2872 ) is issued for a database. There will be exactly one replication controller instance for every database with a master role. It seems to be a good idea to boot the replication controller as a service module on a per need basis. This is, e.g., done to boot the TransactionFactory and DataFactory services in engine/org/apache/derby/impl/store/raw/RawStore.java RawStore.java#boot, line 215-221: ---- 8< ---- xactFactory = (TransactionFactory) Monitor.bootServiceModule( create, this, getTransactionFactoryModule(), properties); dataFactory = (DataFactory) Monitor.bootServiceModule( create, this, getDataFactoryModule(), properties); ---- >8 ---- The module will need an API for these commands: start replication stop replication The controller will be managing the replication strategy (asynchronous, 1-safe, 2-safe, very-safe). For now, only asynchronous is considered. If more strategies are added later, the API should probably be extended with methods to dynamically change the replication strategy. Development on this issue will likely be incremental - the service must be modified as more functionality is in place.

            People

            • Assignee:
              Jørgen Løland
              Reporter:
              Jørgen Løland
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development