Derby
  1. Derby
  2. DERBY-2922 Replication: Add master replication mode
  3. DERBY-3359

The log shipper needs to be modified to vary the shipping intervals dynamically (based on load)

    Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.4.1.3
    • Fix Version/s: 10.4.1.3
    • Component/s: Replication
    • Labels:
      None
    1. DynamicLogShipping_v1.diff
      15 kB
      V.Narayanan
    2. DynamicLogShipping_v1.stat
      0.4 kB
      V.Narayanan
    3. DynamicLogShipping_v2.diff
      12 kB
      V.Narayanan
    4. DynamicLogShipping_v2.stat
      0.4 kB
      V.Narayanan
    5. DynamicLogShipping_v3.diff
      13 kB
      V.Narayanan
    6. DynamicLogShipping_v3.stat
      0.4 kB
      V.Narayanan
    7. DynamicLogShipping_v4.diff
      0.6 kB
      V.Narayanan
    8. DynamicLogShipping_v4.stat
      0.1 kB
      V.Narayanan

      Activity

      Hide
      V.Narayanan added a comment - - edited

      Introduction
      ------------

      The log shipper currently has a static shipping interval of 1 second with
      a forceFlush capability when the buffer becomes full. This algorithm does not
      augur well in the real-time environment and the bond between the buffer fill
      and the log shipper frequency needs to be strengthened to allow shipping
      to depend on incoming transaction load.

      Keeping this in mind the following refined shipping scheme is proposed for the
      first version of replication. If required the design can be later modified to
      a more efficient and complex implementations in later refinements of replication.

      Basic Idea
      ----------

      The Log Buffer will notify the Log Shipper when a buffer is full. When the Log
      Shipper gets this notification it will decide the action to be done the following
      way

      1) It will retrieve the fill information from the Log Buffer. The fill information
      is a indicator of how full the log is at any point of time.

      fill information = (full buffers/Total Buffers)*100

      2) If the fill information is greater than 80 the log shipper will ship log records
      until the fill information returns a value less than 80. There will be no time
      delay between the ships.

      3) If the fill information is less than 80 but greater than 10 the log shipper will
      ship with a MID ms delay. (MID has not been decided yet)

      4) If the fill information is less than 10 the log shipper will ship with a MAX ms delay
      or when a buffer becomes full whichever comes first. The delay however will not be
      smaller than MID.
      (MAX > MID exact values of MAX has not been decided yet).

      Note: MID is a value that is only as large as not to affect the performance of the master database
      significantly.

      max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MID) is the maximum delay between a log record is committed at the master until
      it is replicated to the slave. Maybe we should make sure that MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID.

      Show
      V.Narayanan added a comment - - edited Introduction ------------ The log shipper currently has a static shipping interval of 1 second with a forceFlush capability when the buffer becomes full. This algorithm does not augur well in the real-time environment and the bond between the buffer fill and the log shipper frequency needs to be strengthened to allow shipping to depend on incoming transaction load. Keeping this in mind the following refined shipping scheme is proposed for the first version of replication. If required the design can be later modified to a more efficient and complex implementations in later refinements of replication. Basic Idea ---------- The Log Buffer will notify the Log Shipper when a buffer is full. When the Log Shipper gets this notification it will decide the action to be done the following way 1) It will retrieve the fill information from the Log Buffer. The fill information is a indicator of how full the log is at any point of time. fill information = (full buffers/Total Buffers)*100 2) If the fill information is greater than 80 the log shipper will ship log records until the fill information returns a value less than 80. There will be no time delay between the ships. 3) If the fill information is less than 80 but greater than 10 the log shipper will ship with a MID ms delay. ( MID has not been decided yet ) 4) If the fill information is less than 10 the log shipper will ship with a MAX ms delay or when a buffer becomes full whichever comes first. The delay however will not be smaller than MID. ( MAX > MID exact values of MAX has not been decided yet ). Note: MID is a value that is only as large as not to affect the performance of the master database significantly. max(MAX, DEFAULT_NUMBER_LOG_BUFFERS*MID) is the maximum delay between a log record is committed at the master until it is replicated to the slave. Maybe we should make sure that MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID.
      Hide
      Jørgen Løland added a comment -

      Thank you for the very clear description of the issue, Narayanan. This looks like a good idea to me.

      A question: How will MAX and MID be set? Do you intend for these to be hardcoded constants, properties that can be decided by the user or something else?

      Show
      Jørgen Løland added a comment - Thank you for the very clear description of the issue, Narayanan. This looks like a good idea to me. A question: How will MAX and MID be set? Do you intend for these to be hardcoded constants, properties that can be decided by the user or something else?
      Hide
      V.Narayanan added a comment -

      >How will MAX and MID be set? Do you intend for these to be hardcoded constants, properties that can be decided by the user or >something else?

      I expect them to be harcoded constants for the first version of the replication release.
      An improvement would be to make MID user configurable.

      MAX can be derived from this I guess? (Maybe we should make sure that
      MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID ref. spec above).

      For the first release (atleast for the first version of this patch) I was hoping that if I do not get
      any suggestions I will use a value of 50ms for MID.

      Show
      V.Narayanan added a comment - >How will MAX and MID be set? Do you intend for these to be hardcoded constants, properties that can be decided by the user or >something else? I expect them to be harcoded constants for the first version of the replication release. An improvement would be to make MID user configurable. MAX can be derived from this I guess? (Maybe we should make sure that MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID ref. spec above). For the first release (atleast for the first version of this patch) I was hoping that if I do not get any suggestions I will use a value of 50ms for MID.
      Hide
      V.Narayanan added a comment -

      Implementation Nuances
      ----------------------

      Here, I will be explaining the changes that will be required to the appropriate classes for
      implementating the Dynamic log shipping functionality as detailed above.

      org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer
      ----------------------------------------------------------------------

      This class will contain the functionality for

      1) calculating the Fill Information (FI)
      2) For notifying the log shipper each time a log buffer element becomes full.

      1) calculating the Fill Information (FI)
      ------------------------------------------

      FI will be an integer with a value ranging between 0-100

      FI = (dirtyBuffers.size()/DEFAULT_NUMBER_LOG_BUFFERS)*100

      where,

      dirtyBuffers is a LinkedList containing the log buffers elements with unsent log.

      DEFAULT_NUMBER_LOG_BUFFERS The total number of log buffers elements (filled and unfilled).

      A method,

      public int getFillInformation();

      will be added that will return the value of FI as calculated above.

      2) For notifying the log shipper each time a log buffer element becomes full.
      ----------------------------------------------------------------------------

      Each time a log shipper element becomes full we call switchDirtyBuffer() in order to
      Append the currentDirtyBuffer to dirtyBuffers, and make a fresh buffer element from
      freeBuffers.

      We will add logic in the switchDirtyBuffer() method for notifying the Log Shipper.

      For more information on the LogShipper please refer to,
      1) http://issues.apache.org/jira/browse/DERBY-2926
      2) http://issues.apache.org/jira/browse/DERBY-3051

      org.apache.derby.impl.services.replication.master.LogShipper (interface)
      ------------------------------------------------------------------------

      This interface will contain the definition of the method that will be called by the
      log buffer.

      Will add a method workToDo that will be used by the Log Buffer to inform the log shipper
      whenever a Log Buffer Element becomes full.

      org.apache.derby.impl.services.replication.master.AsynchronousLogShipper
      ------------------------------------------------------------------------

      This class will contain the functionality for managing load in the Log buffer (by shipping log records)
      ,by using the FI information, during each buffer element full notification that is received.

      Two constants
      -------------

      MID = 50ms
      MAX = 550ms (MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID)

      attribute nextShippingTime
      --------------------------

      will store the time at which the next log ship will happen and will be calculated as System.currentTimeMillis() +
      shippingInterval in the run method.

      methods
      -------

      constructor
      -----------

      shippingInterval will be initialized to MID in the beginning.

      nextShippingTime will be initialized in the constructor initially to
      System.currentTimeMills() + shippingInterval (i.e. MID).

      public void workToDo();
      -----------------------

      Will be added that will be called by the log buffer each time a buffer element becomes full.

      The following steps will be used in this method

      a) Get FI from log buffer
      b) If FI >= 80
      b.1) call shipALogChunk()
      b.2) repeat b) until FI < 80
      b.3) notify the log shipper thread.
      c) If FI > 10 && FI < 80
      c.1) calculate the time elapsed since last log ship.
      (For this before calling wait we need to store the
      time at which next transmit will happen.)
      c.2) If the time remaining for the next ship is less than
      MID then do not do anything.
      c.3) If the time elapsed since last ship is greater than
      MID notify the log shipper thread. (here we could have
      simply not shipped and waited for timeout to happen, but
      this would be a risk and a better option would be to ship)

      will check if the forceFlush method is required at all since all of the work done by the forceFlush
      method can now be done by the workToDo method (see changes to MasterController).

      public void calculateWTfromFI() (WT - wait time)
      -------------------------------

      will be used to calculate the wait time using the FI from the buffer.

      shippingInterval will be assigned to the return value from this method before calling
      wait(shippingInterval) in the run method. *If this method returns -1 however we should
      not call wait but just continue with the shipping.*

      The following steps will be used in this method

      a) FI >= 80 return -1 (we should not call wait if the return value is -1)
      b) FI > 10 and FI < 80 return MID
      c) FI <= 10 return MAX

      run()


      This method will be modified to

      1) shippingInterval will be assigned the return value
      of calculateWTfromFI().
      2) calculate the next shipping time which will be
      System.currentTimeMillis() + shippingInterval.

      For more information on the log shipper please refer to
      http://issues.apache.org/jira/browse/DERBY-3064

      org.apache.derby.impl.services.replication.master.MasterController
      ------------------------------------------------------------------

      When a LogBufferFullException is thrown we will call workToDo instead of forceFlush.

      For more information on the MasterController please refer to
      http://issues.apache.org/jira/browse/DERBY-2977

      Show
      V.Narayanan added a comment - Implementation Nuances ---------------------- Here, I will be explaining the changes that will be required to the appropriate classes for implementating the Dynamic log shipping functionality as detailed above. org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer ---------------------------------------------------------------------- This class will contain the functionality for 1) calculating the Fill Information ( FI ) 2) For notifying the log shipper each time a log buffer element becomes full. 1) calculating the Fill Information ( FI ) ------------------------------------------ FI will be an integer with a value ranging between 0-100 FI = (dirtyBuffers.size()/DEFAULT_NUMBER_LOG_BUFFERS)*100 where, dirtyBuffers is a LinkedList containing the log buffers elements with unsent log. DEFAULT_NUMBER_LOG_BUFFERS The total number of log buffers elements (filled and unfilled). A method, public int getFillInformation(); will be added that will return the value of FI as calculated above. 2) For notifying the log shipper each time a log buffer element becomes full. ---------------------------------------------------------------------------- Each time a log shipper element becomes full we call switchDirtyBuffer() in order to Append the currentDirtyBuffer to dirtyBuffers, and make a fresh buffer element from freeBuffers. We will add logic in the switchDirtyBuffer() method for notifying the Log Shipper. For more information on the LogShipper please refer to, 1) http://issues.apache.org/jira/browse/DERBY-2926 2) http://issues.apache.org/jira/browse/DERBY-3051 org.apache.derby.impl.services.replication.master.LogShipper (interface) ------------------------------------------------------------------------ This interface will contain the definition of the method that will be called by the log buffer. Will add a method workToDo that will be used by the Log Buffer to inform the log shipper whenever a Log Buffer Element becomes full. org.apache.derby.impl.services.replication.master.AsynchronousLogShipper ------------------------------------------------------------------------ This class will contain the functionality for managing load in the Log buffer (by shipping log records) ,by using the FI information, during each buffer element full notification that is received. Two constants ------------- MID = 50ms MAX = 550ms (MAX > DEFAULT_NUMBER_LOG_BUFFERS*MID) attribute nextShippingTime -------------------------- will store the time at which the next log ship will happen and will be calculated as System.currentTimeMillis() + shippingInterval in the run method. methods ------- constructor ----------- shippingInterval will be initialized to MID in the beginning. nextShippingTime will be initialized in the constructor initially to System.currentTimeMills() + shippingInterval (i.e. MID). public void workToDo(); ----------------------- Will be added that will be called by the log buffer each time a buffer element becomes full. The following steps will be used in this method a) Get FI from log buffer b) If FI >= 80 b.1) call shipALogChunk() b.2) repeat b) until FI < 80 b.3) notify the log shipper thread. c) If FI > 10 && FI < 80 c.1) calculate the time elapsed since last log ship. (For this before calling wait we need to store the time at which next transmit will happen.) c.2) If the time remaining for the next ship is less than MID then do not do anything. c.3) If the time elapsed since last ship is greater than MID notify the log shipper thread. (here we could have simply not shipped and waited for timeout to happen, but this would be a risk and a better option would be to ship) will check if the forceFlush method is required at all since all of the work done by the forceFlush method can now be done by the workToDo method (see changes to MasterController). public void calculateWTfromFI() (WT - wait time) ------------------------------- will be used to calculate the wait time using the FI from the buffer. shippingInterval will be assigned to the return value from this method before calling wait(shippingInterval) in the run method. *If this method returns -1 however we should not call wait but just continue with the shipping.* The following steps will be used in this method a) FI >= 80 return -1 ( we should not call wait if the return value is -1 ) b) FI > 10 and FI < 80 return MID c) FI <= 10 return MAX run() This method will be modified to 1) shippingInterval will be assigned the return value of calculateWTfromFI(). 2) calculate the next shipping time which will be System.currentTimeMillis() + shippingInterval. For more information on the log shipper please refer to http://issues.apache.org/jira/browse/DERBY-3064 org.apache.derby.impl.services.replication.master.MasterController ------------------------------------------------------------------ When a LogBufferFullException is thrown we will call workToDo instead of forceFlush. For more information on the MasterController please refer to http://issues.apache.org/jira/browse/DERBY-2977
      Hide
      Jørgen Løland added a comment -

      Thanks for writing such a detailed implementation suggestion, Narayanan. This should make it easy to discuss the suggested solution and review the code later!

      I have only one concern with this, regarding workToDo case b):

      Basically, what you're saying, is that if the log becomes 80% full, the client thread should be used to ship log records. This choice has at least the following consequences:

      • The log buffer can never get fuller than 90% (with the current 10 LogBufferElements)
      • We start increasing the client response time even in cases where the log produced by the transaction that client is involved in would fit in the buffer.

      Based on this, I think workToDo case b) could simply be reduced to on b.3, i.e. notify the log shipper thread that it has to wake up no matter what. By doing that, we would still need the forceFlush method, but the client thread would not be involved in log shipping until the buffer is 100% full.

      Show
      Jørgen Løland added a comment - Thanks for writing such a detailed implementation suggestion, Narayanan. This should make it easy to discuss the suggested solution and review the code later! I have only one concern with this, regarding workToDo case b): Basically, what you're saying, is that if the log becomes 80% full, the client thread should be used to ship log records. This choice has at least the following consequences: The log buffer can never get fuller than 90% (with the current 10 LogBufferElements) We start increasing the client response time even in cases where the log produced by the transaction that client is involved in would fit in the buffer. Based on this, I think workToDo case b) could simply be reduced to on b.3, i.e. notify the log shipper thread that it has to wake up no matter what. By doing that, we would still need the forceFlush method, but the client thread would not be involved in log shipping until the buffer is 100% full.
      Hide
      V.Narayanan added a comment -

      >Based on this, I think workToDo case b) could simply be reduced
      >to on b.3, i.e. notify the log shipper thread that it has to wake up
      >no matter what. By doing that, we would still need the forceFlush
      >method, but the client thread would not be involved in log shipping
      >until the buffer is 100% full.

      makes sense to notify the log shipper thread and let it handle the log shipping.
      I agree with your comments Jorgen! I will remove b.1 and b.2 and shall retain
      forceFlush.

      Show
      V.Narayanan added a comment - >Based on this, I think workToDo case b) could simply be reduced >to on b.3, i.e. notify the log shipper thread that it has to wake up >no matter what. By doing that, we would still need the forceFlush >method, but the client thread would not be involved in log shipping >until the buffer is 100% full. makes sense to notify the log shipper thread and let it handle the log shipping. I agree with your comments Jorgen! I will remove b.1 and b.2 and shall retain forceFlush.
      Hide
      V.Narayanan added a comment -

      Pls find attached the patch for the implementation of dynamic log shipping. I have to test this code
      for various cases but did not see that as stopping a review of this patch since the design,
      and the subsequent changes required have been described in detail in previous comments on this patch.

      I thought that the workToDo() method can be used in the place of the forceFlush() method. When a
      LogBufferFullException occurs in the MasterController the FI > 80 so this method would automatically
      result in shipALogChunk() being called. The only difference from calling forceFlush() being that we do an
      additional checking of FI before shipping. I am not sure this should be a deterrent to using this method.
      Consequently,

      • Removed the use of forceFlush() when a LogBufferFullException is thrown and replaced it with workToDo().
      • Removed the forceFlush() method from the LogBuffer interface and its AsynchronousLogShipper implementation
        as it will not be used anymore.
      • workToDo() will now be called both when a log buffer element becomes full and when the log buffer becomes
        full
      Show
      V.Narayanan added a comment - Pls find attached the patch for the implementation of dynamic log shipping. I have to test this code for various cases but did not see that as stopping a review of this patch since the design, and the subsequent changes required have been described in detail in previous comments on this patch. I thought that the workToDo() method can be used in the place of the forceFlush() method. When a LogBufferFullException occurs in the MasterController the FI > 80 so this method would automatically result in shipALogChunk() being called. The only difference from calling forceFlush() being that we do an additional checking of FI before shipping. I am not sure this should be a deterrent to using this method. Consequently, Removed the use of forceFlush() when a LogBufferFullException is thrown and replaced it with workToDo(). Removed the forceFlush() method from the LogBuffer interface and its AsynchronousLogShipper implementation as it will not be used anymore. workToDo() will now be called both when a log buffer element becomes full and when the log buffer becomes full
      Hide
      V.Narayanan added a comment -

      On retrospection the forceFlush is required, it would be very
      bad for a method that is called upon receiving a LogBufferFullException
      to exit without shipping log and freeing space in the log buffer.
      It is OK for the log shipper thread to take over from where the
      forceFlush thread leaves.

      I will post this patch again with the forceFlush method!

      Show
      V.Narayanan added a comment - On retrospection the forceFlush is required, it would be very bad for a method that is called upon receiving a LogBufferFullException to exit without shipping log and freeing space in the log buffer. It is OK for the log shipper thread to take over from where the forceFlush thread leaves. I will post this patch again with the forceFlush method!
      Hide
      V.Narayanan added a comment -

      Modified v1 to switch back to using forceFlush!

      Show
      V.Narayanan added a comment - Modified v1 to switch back to using forceFlush!
      Hide
      Øystein Grøvlen added a comment -

      Thanks, for the patch Narayanan. I think I have convinced myself that
      the code would work, but I am not sure that it matches the description
      of the basic idea above. The description says:

      > If the fill information is less than 10 the log shipper will ship
      > with a MAX ms delay or when a buffer becomes full whichever comes
      > first.

      I do not think this is what is implemented. If there is 10 or more
      log buffers, it seems workToDo() will do nothing when the first log
      buffer goes full. I would consider reasoning around lastShippingTime
      instead of nextShippingTime. I think that would also make the
      logic easier to understand. In other words, I suggest something like
      this in workToDo():

      if (fi >= 80)

      { notify(); }

      else if (System.currentTimeMillis() - lastShippingTime) > MID)

      { // Minimum MID time between messages unless buffer is almost full notify(); }

      Some minor things:

      1. It seems to me that MIN would be a more descriptive name than MID.
      It is the minimum interval when not in "Panic mode".

      2. I suggest increasing MID to 100.

      3. The MAX value could be higher. I would think a default latency of
      5 seconds should be acceptable.

      4. The last sentence of Javadoc for MAX needs an explanation.

      5. ReplicationLogBuffer: Why add a new constructor? I think it would
      be better to just add a parameter to the existing. Having the
      original constructor, opens for objects that will give null pointer
      exceptions where mf is referred.

      6. Comments in ReplicationLogBuffer refers to LogShipper, but
      strictly speaking ReplicationLogBuffer does not know of any
      LogShipper, just that the MasterFactory wants to know when there is
      more work to be done.

      7. I think you should use final fields for the thresholds (10/80). That
      would make them easier to change if necessary.

      Show
      Øystein Grøvlen added a comment - Thanks, for the patch Narayanan. I think I have convinced myself that the code would work, but I am not sure that it matches the description of the basic idea above. The description says: > If the fill information is less than 10 the log shipper will ship > with a MAX ms delay or when a buffer becomes full whichever comes > first. I do not think this is what is implemented. If there is 10 or more log buffers, it seems workToDo() will do nothing when the first log buffer goes full. I would consider reasoning around lastShippingTime instead of nextShippingTime. I think that would also make the logic easier to understand. In other words, I suggest something like this in workToDo(): if (fi >= 80) { notify(); } else if (System.currentTimeMillis() - lastShippingTime) > MID) { // Minimum MID time between messages unless buffer is almost full notify(); } Some minor things: 1. It seems to me that MIN would be a more descriptive name than MID. It is the minimum interval when not in "Panic mode". 2. I suggest increasing MID to 100. 3. The MAX value could be higher. I would think a default latency of 5 seconds should be acceptable. 4. The last sentence of Javadoc for MAX needs an explanation. 5. ReplicationLogBuffer: Why add a new constructor? I think it would be better to just add a parameter to the existing. Having the original constructor, opens for objects that will give null pointer exceptions where mf is referred. 6. Comments in ReplicationLogBuffer refers to LogShipper, but strictly speaking ReplicationLogBuffer does not know of any LogShipper, just that the MasterFactory wants to know when there is more work to be done. 7. I think you should use final fields for the thresholds (10/80). That would make them easier to change if necessary.
      Hide
      V.Narayanan added a comment -

      Thank you for the comments/reviews on the patch Oystein!

      I have done all the changes as suggested by you.

      Some additional changes are enumerated and explained below,

      • Changed the Class level comment to indicate that the shipping happens
        when notification is received from the log buffer also. Also removed
        some comments that are not valid anymore.
      • lastShippingTime is initialized to System.currentTimemillis() in the
        beginning. This would mean that the first shipping for medium load
        FI_LOW < fi < FI_HIGH cannot happen unless MIN ms has elapsed.
      • lastShippingTime is initialized in the shipALogChunk() method.
      Show
      V.Narayanan added a comment - Thank you for the comments/reviews on the patch Oystein! I have done all the changes as suggested by you. Some additional changes are enumerated and explained below, Changed the Class level comment to indicate that the shipping happens when notification is received from the log buffer also. Also removed some comments that are not valid anymore. lastShippingTime is initialized to System.currentTimemillis() in the beginning. This would mean that the first shipping for medium load FI_LOW < fi < FI_HIGH cannot happen unless MIN ms has elapsed. lastShippingTime is initialized in the shipALogChunk() method.
      Hide
      Øystein Grøvlen added a comment -

      Thanks for addressing my comments, Narayanan.
      Patch v3 commited as revision 620453.

      Show
      Øystein Grøvlen added a comment - Thanks for addressing my comments, Narayanan. Patch v3 commited as revision 620453.
      Hide
      Jørgen Løland added a comment -

      I tried running replication with the new log shipper. Seems mf is never initialized in ReplicationLogBuffer.

      Master-side start replication (slave side behaves normally):

      $ ij
      ij version 10.4
      ij> connect 'jdbc:derby:t1';
      ij> connect 'jdbc:derby:t1;startMaster=true;slaveHost=localhost';
      ij(CONNECTION1)> Exception in thread "Thread-1" java.lang.NullPointerException
      at org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer.switchDirtyBuffer(ReplicationLogBuffer.java:319)
      at org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer.next(ReplicationLogBuffer.java:187)
      at org.apache.derby.impl.services.replication.master.AsynchronousLogShipper.shipALogChunk(AsynchronousLogShipper.java:213)
      at org.apache.derby.impl.services.replication.master.AsynchronousLogShipper.run(AsynchronousLogShipper.java:165)

      Show
      Jørgen Løland added a comment - I tried running replication with the new log shipper. Seems mf is never initialized in ReplicationLogBuffer. Master-side start replication (slave side behaves normally): $ ij ij version 10.4 ij> connect 'jdbc:derby:t1'; ij> connect 'jdbc:derby:t1;startMaster=true;slaveHost=localhost'; ij(CONNECTION1)> Exception in thread "Thread-1" java.lang.NullPointerException at org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer.switchDirtyBuffer(ReplicationLogBuffer.java:319) at org.apache.derby.impl.services.replication.buffer.ReplicationLogBuffer.next(ReplicationLogBuffer.java:187) at org.apache.derby.impl.services.replication.master.AsynchronousLogShipper.shipALogChunk(AsynchronousLogShipper.java:213) at org.apache.derby.impl.services.replication.master.AsynchronousLogShipper.run(AsynchronousLogShipper.java:165)
      Hide
      V.Narayanan added a comment -

      Thank you for the commit Oystein

      Show
      V.Narayanan added a comment - Thank you for the commit Oystein
      Hide
      V.Narayanan added a comment -

      sorry my mistake . Pls find patch attached that fixes this issue

      Show
      V.Narayanan added a comment - sorry my mistake . Pls find patch attached that fixes this issue
      Hide
      Øystein Grøvlen added a comment -

      Committed patch v4 as revision 620483.
      Checked that starting replication works.

      Show
      Øystein Grøvlen added a comment - Committed patch v4 as revision 620483. Checked that starting replication works.
      Hide
      Knut Anders Hatlen added a comment -

      Could we also make mf final? Then we would have caught the error at compile time.

      Show
      Knut Anders Hatlen added a comment - Could we also make mf final? Then we would have caught the error at compile time.
      Hide
      Henri van de Scheur added a comment -

      > Knut Anders Hatlen commented on DERBY-3359:
      >
      > Could we also make mf final? Then we would have caught the error at compile time.

      Good proposal. Fix committed as revision 620499.

      Show
      Henri van de Scheur added a comment - > Knut Anders Hatlen commented on DERBY-3359 : > > Could we also make mf final? Then we would have caught the error at compile time. Good proposal. Fix committed as revision 620499.
      Hide
      Knut Anders Hatlen added a comment -

      The workToDo() method added in v3 seems to have a bug, see DERBY-3406. It should probably have a synchronized block around the calls to notify().

      Show
      Knut Anders Hatlen added a comment - The workToDo() method added in v3 seems to have a bug, see DERBY-3406 . It should probably have a synchronized block around the calls to notify().
      Hide
      V.Narayanan added a comment -

      Thank you for pointing this out. I will produce a fix soon.

      Show
      V.Narayanan added a comment - Thank you for pointing this out. I will produce a fix soon.
      Hide
      V.Narayanan added a comment -

      All relevant patches have been committed. Resolving issue!

      Show
      V.Narayanan added a comment - All relevant patches have been committed. Resolving issue!

        People

        • Assignee:
          V.Narayanan
          Reporter:
          V.Narayanan
        • Votes:
          0 Vote for this issue
          Watchers:
          0 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development