I think this is a regression. I've tested that NetworkServerControl from 10.4.1.3, 10.4.2.0 and 10.5.1.1 can't talk to a 10.3.3.0 (or earlier) server. Tried runtimeinfo, ping and shutdown.

If we decide that this regression is acceptable (the workaround is simple enough: don't use derbynet.jar>=10.4 to talk to server<10.4) we should at least make the compatibility tests run cleanly.

Triaged for 10.5.2.

It does look like we keep a PROTOCOL_VERSION in NetworkServerControlImpl and it does look like this was bumped with DERBY-2109, linked to this issue, so hopefully this is fixable with conditional logic around the version.

I debugged the code a little and found that the reason behind the errors we see is that with DERBY-2109 , the protocol version has been bumped up to reflect the change in the syntax of shutdown. During the handshake between client and server, client writes down what protocol version it has. The server on the receiving end finds that the protocol does not match and it throws an exception, This is why, even though, DERBY-2109 only changed shutdown, we see the impact for commands like ping too.

I experimented by resetting the protocol version back to it's original value prior to DERBY-2109. This ofcourse fixes commands like ping coming from trunk to 10.3 server because now server and client are the same protocol version. But this does impact shutdown. Following is the observation with shutdown.
There are 2 possible scenarios for client and server
1)Network Server started with 10.3 codeline(does not have DERBY-2109 fix) and client running with trunk codeline(does have DERBY-2109 fix)
2)Network Server started with trunk and client running with 10.3 codeline

The current behavior of the codelines for the above 2 cases is as follows(ie when trunk is at higher protocol version than 10.3)
1)Gives error on both client and server side
Client error : Thu Aug 26 11:27:45 PDT 2010 : Invalid reply header from network server: Invalid string Plaintext connection attempt to an SSL enabled server?
Server error : DRDA_UnknownProtocol.S, <2> java.lang.Throwable: DRDA_UnknownProtocol.S, <2>
2)10,3 client hangs when shutdown is issues by it. Ctrl-C out of that client and get following error on server side
Server error : Execution failed because of a Distributed Protocol Error: DRDA_Disconnect; CODPNT arg = 0; Error Code Value = 0,DDMReader.fill(),InputStream.read(),Connection reset,*

With the experiment I mentioned above, ie the trunk does not change the protocol level and stays at the same level as DERBY-2109. With this change, the above 2 cases behave as follows
1)No error any more for this case because both client and server are at the same protocol level
2)This case contrinues to behave the same, ie 10.3 client hangs when it issues shutdown for network server started by trunk codeline with my changes
Ctrl-C out of that client and get following error on server side
Server error : Execution failed because of a Distributed Protocol Error: DRDA_Disconnect; CODPNT arg = 0; Error Code Value = 0,DDMReader.fill(),InputStream.read(),Connection reset,*

BTW, I tested all the options on NetworkServerControl as listed below after resetting the protocol level and everything with the exception of shutdown(more infor on shutdown in the earlier paragraphs) worked fine with server at 10.3 level and client at trunk and vice versa

;

We could go ahead and change the protocol back to pre-DERBY-2109 level because even though DERBY-2109 bumped up the protocol, it didn't provide any work to ensure the compatibility between different protocol versions.
2 possible solutions I think can of at this point are
1)Revert the protocol back to pre-DERBY-2109. This fixes DERBY-3644 and improves the behavior for Network Server with 10,3 and shutdown coming from trunk. This work is obviosly very easy.
2)Change the trunk to be able to handle different protocol level rather than throw error when non-matching protocol is detected. This work is more involved and will not fix DERBY-3644 for existing released products.

Thank you Mamta for looking at this issue. It would seem the PROTOCOL_VERSION is not very helpful with a newer client because there is no handshake in the protocol to negotiate down. I guess in retrospect it would have been better to just have a new COMMAND_SHUTDOWN_USER_PASS command for DERBY-2109 and leave the old one alone, but I think to fix this issue maybe the following approach might work.

1) Send protocol version 1 for all the unchanged commands.
2) For shutdown, If no user/password are specified, first do a ping which will get a reply header from the server so we can see its protocol version. Then send version 1 (no user/pass) or version 2 (user/pass) as appropriate.
3) If user/password are specified, just send version 2 format.
4) Change COMMAND_SHUTDOWN parsing to do the right thing for protocol version 1 vs 2.

Shutdown will have an extra round trip, but I think all the commands should work.

I will look further but from my quick look at the code, I do not think server sends the protocol version as part of the handshake for a ping. I will spend little more time investigating.

You are right. The reply header doesn't have the version. Well then, I don't have any clever ideas to get shutdown to work. I think I would start by checking in a fix to send version 1 for the other commands and leave shutdown at 2.
Then this issue could be closed and a new one opened up for the shutdown issue and we could then look at how messy it would be to try to fix. Perhaps we could send at level 2 and then retry at level 1 if we get the invalidReplyHeader error.

Just commited a change into trunk with revision 990841 which has reverted back the prtocol level back to 1. I will create another jira for shutdown issue and look into that.

the commit comments were as follows
DERBY-3644 and DERBY-2109

As part of DERBY-2109, shutdown command started accepting optional user name and password. Bacause of this change in command, the protcol level was bumped up to 2. But this caused all the network server commands from clients with DERBY-2109 changes to break against a server without the DERBY-2109 changes. That is because the server checks the protocol level of the client commands and if they do match with server protocol version, then server will simply throw an exception about mistmatch of protocol levels and thus causing all the client commands from post DERBY-2109 clients to break against server with pre-DERBY-2109 changes.

Will create another jira to handle the shutdown command's optional user name and password parameters.

I forgot to mention that I ran derbyall succesfully with my changes. Junit suite run had following 3 failures both with and without my changes

1) testPing(org.apache.derbyTesting.functionTests.tests.derbynet.NetworkServerControlClientCommandTest)junit.framework.AssertionFailedError
at org.apache.derbyTesting.junit.BaseTestCase.assertExecJavaCmdAsExpected(BaseTestCase.java:484)
at org.apache.derbyTesting.functionTests.tests.derbynet.NetworkServerControlClientCommandTest.assertFailedPing(NetworkServerControlClientCommandTest.java:116)
at org.apache.derbyTesting.functionTests.tests.derbynet.NetworkServerControlClientCommandTest.testPing(NetworkServerControlClientCommandTest.java:78)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:23)
at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:23)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:23)
2) testInvalidLDAPServerConnectionError(org.apache.derbyTesting.functionTests.tests.jdbcapi.InvalidLDAPServerAuthenticationTest)junit.framework.AssertionFailedError
at org.apache.derbyTesting.functionTests.tests.jdbcapi.InvalidLDAPServerAuthenticationTest.testInvalidLDAPServerConnectionError(InvalidLDAPServerAuthenticationTest.java:122)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:16)
at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:16)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:16)
3) testInvalidLDAPServerConnectionErrorjunit.framework.AssertionFailedError: C:\p4clients\svnmain\client2\trunk\systest\out142\system\singleUse\oneuse48\db.lck
at org.apache.derbyTesting.junit.DropDatabaseSetup.removeDir(DropDatabaseSetup.java:116)
at org.apache.derbyTesting.junit.DropDatabaseSetup.access$000(DropDatabaseSetup.java:35)
at org.apache.derbyTesting.junit.DropDatabaseSetup$1.run(DropDatabaseSetup.java:105)
at java.security.AccessController.doPrivileged(AccessController.java:202)
at org.apache.derbyTesting.junit.DropDatabaseSetup.removeDirectory(DropDatabaseSetup.java:102)
at org.apache.derbyTesting.junit.DropDatabaseSetup.removeDirectory(DropDatabaseSetup.java:98)
at org.apache.derbyTesting.junit.DropDatabaseSetup.removeDatabase(DropDatabaseSetup.java:91)
at org.apache.derbyTesting.junit.TestConfiguration$3.tearDown(TestConfiguration.java:710)
at junit.extensions.TestSetup$1.protect(TestSetup.java:20)
at junit.extensions.TestSetup.run(TestSetup.java:16)
at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:16)
at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
at junit.extensions.TestSetup.run(TestSetup.java:16)

FAILURES!!!
Tests run: 12836, Failures: 3, Errors: 0

assigning to Mamta as she already made a commit for this issue

This jira should not be backported to earlier releases without the accompanying changes for DERBY-4786. This jira has reverted the protocol level to 1 for all the commands. DERBY-4786 will make sure that we recognize that shutdown is at higher protocol level because it has been changed to allow optional user credentials.

This jira has been fixed as part of the checkin for DERBY-4786. It has been backporting to 10.7 and 10.6 as of now. Working on backporting to 10.5 and 10.4

I don't think the fix made it into the 10.6 branch until after the 10.6.2.1 release candidate was produced, so I've changed the fix-version to 10.6.2.2.

Thanks for fixing the 10.6 version, Knut. I must not have synced my client when I did sysinfo on it.

[bulk update] Close all resolved issues that haven't been updated for more than one year.