[DERBY-3639] Slave on Windows stops replication when network connection is broken, failover fails

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 10.4.1.3
• Component/s: Replication
• Labels: derby_triage10_5_2
• Environment: Master on Solaris 10 x86, Slave on Windows XP SP2.; Slave VM: Sun's Java HotSpot Client VM build 1.6.0_03-b05.
• Urgency: Normal
• Issue & fix info: Workaround attached
• Bug behavior facts: Crash
• Assignee: Unassigned
• Reporter: John Embretsen
• Created: 22/Apr/08 13:46
• Updated: 18/Jan/13 09:12

Description

Replication: Failover on slave fails after network connection is broken (network cable to slave pulled out); database is shut down due to an "unexpected error".
Same experiment with a replicated embedded database on Linux (FC5) and Windows resulted in success on Linux and failure on Windows.

Documentation (admin guide, "Replication failure handling") says:

"Slave loses connection with master: The slave tries to reestablish the connection with the master by listening on the specified host and port. It will not give up until it is explicitly requested to do so by either the failover=true or stopSlave=true connection URL attribute. If a failover is requested, the slave applies all received log records and boots the database as described in Forcing a failover."

Slave console:

java -jar lib\derbyrun.jar ij

ij version 10.4

ij> connect 'jdbc:derby:replicDB;startSlave=true;slaveHost=0.0.0.0';

ERROR XRE08: Replication slave mode started successfully for database 'replicDB'. Connection refused because the database is in replication slave mode

.

ij> connect 'jdbc:derby:replicDB';

ERROR 08004: Connection refused to database 'replicDB' because it is in replication slave mode.

ij> – network cable unplugged from slave

ij> connect 'jdbc:derby:replicDB;failover=true';

ERROR XRE11: Could not perform operation 'failover' because the database 'replicDB' has not been booted.

ij> connect 'jdbc:derby:replicDB';

ij>

The slave's derby.log reported the following after the network cable was pulled and the failover command was issued:

---- BEGIN REPLICATION ERROR MESSAGE (4/22/08 2:01 PM) ----

Replication slave got a fatal error for database 'replicDB'. Replication will be stopped.

ERROR XRE03: Unexpected replication error. See derby.log for details.

at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)

at org.apache.derby.impl.store.replication.slave.SlaveController$SlaveLogReceiverThread.run(Unknown Source)

Caused by: java.net.SocketException: Connection reset

at java.net.SocketInputStream.read(Unknown Source)

at java.net.SocketInputStream.read(Unknown Source)

at java.io.ObjectInputStream$PeekInputStream.peek(Unknown Source)

at java.io.ObjectInputStream$BlockDataInputStream.peek(Unknown Source)

at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)

at java.io.ObjectInputStream.readObject0(Unknown Source)

at java.io.ObjectInputStream.readObject(Unknown Source)

at org.apache.derby.impl.store.replication.net.SocketConnection.readMessage(Unknown Source)

at org.apache.derby.impl.store.replication.net.ReplicationMessageReceive.readMessage(Unknown Source)

[snipped further stack traces]

-------------------- END REPLICATION ERROR MESSAGE ---------------------

Replication slave role was stopped for database 'replicDB'.

Replication slave role was stopped for database 'replicDB'.

2008-04-22 12:01:42.921 GMT:
Shutting down instance 601a400f-0119-7600-eb23-000000383460

----------------------------------------------------------------

Full derby.log from the slave is attached.

Issue Links

• relates to: DERBY-3527 The slave will not notice that a network cable is unplugged and will therefore reject failover/stopSlave commands