Derby
  1. Derby
  2. DERBY-3069

Derby does not resolve functions bound to methods with varargs.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.0.2.0, 10.0.2.1, 10.1.1.0, 10.1.2.1, 10.1.3.1, 10.2.1.6, 10.2.2.0, 10.3.1.4
    • Fix Version/s: 10.10.1.1
    • Component/s: SQL
    • Urgency:
      Normal
    • Issue & fix info:
      Patch Available

      Description

      Varargs were added in Java 5. It would be nice if Derby let you invoke a function bound to a method with a variable length argument list. The Reference Guide states a small number of restrictions for methods which can be invoked as Derby functions: They must be public, static, and not have arguments which are long datatypes. I see no reason that Derby shouldn't be able to resolve and invoke functions which are bound to methods which don't suffer these limitations but which have variable argument lists.

      1. derby-3069-09-aa-docNotSupportedOnJSR169.diff
        0.4 kB
        Rick Hillegas
      2. derby-3069-08-aa-stringArrayTable.diff
        3 kB
        Rick Hillegas
      3. Varargs.html
        10 kB
        Rick Hillegas
      4. derby-3069-07-ab-datatypeTests.diff
        35 kB
        Rick Hillegas
      5. derby-3069-07-aa-datatypeTests.diff
        35 kB
        Rick Hillegas
      6. derby-3069-06-ab-fixBugs.diff
        27 kB
        Rick Hillegas
      7. derby-3069-05-aa-executeVarargs.diff
        44 kB
        Rick Hillegas
      8. derby-3069-04-aa-shortenRoutineNamesInUpgradeTest.diff
        2 kB
        Rick Hillegas
      9. derby-3069-03-ab-varargsSyntax.diff
        57 kB
        Rick Hillegas
      10. derby-3069-03-aa-varargsSyntax.diff
        53 kB
        Rick Hillegas
      11. Varargs.html
        10 kB
        Rick Hillegas
      12. derby-3069-02-backout.diff
        12 kB
        Rick Hillegas
      13. derby-3069-01-varargs-ab.diff
        12 kB
        Rick Hillegas
      14. derby-3069-01-varargs-aa.diff
        11 kB
        Rick Hillegas
      15. z.sql
        0.6 kB
        Rick Hillegas
      16. z.java
        0.8 kB
        Rick Hillegas

        Issue Links

          Activity

          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-09-aa-docNotSupportedOnJSR169.diff. This patch makes a trivial change to the Reference Guide. It adds user-defined aggregates and varargs routines to the list of Derby features which are not supported on JSR 169 platforms. Committed at subversion revision 1439893.

          Touches the following file:

          M src/ref/rrefjdbcjsr169.dita

          Show
          Rick Hillegas added a comment - Attaching derby-3069-09-aa-docNotSupportedOnJSR169.diff. This patch makes a trivial change to the Reference Guide. It adds user-defined aggregates and varargs routines to the list of Derby features which are not supported on JSR 169 platforms. Committed at subversion revision 1439893. Touches the following file: M src/ref/rrefjdbcjsr169.dita
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-08-aa-stringArrayTable.diff. This patch adds a VTI which I find useful for ad-hoc testing of VTIs. Committed at revision 1431623.

          Here is the javadoc for the VTI:

          /**

          • <p>
          • This is a table function which creates a StringArrayVTI out of
          • a space separated list of column names, and a varargs of rows.
          • Each row is a space separated list of column values. Here is
          • a sample usage:
          • </p>
            *
          • <pre>
          • connect 'jdbc:derby:memory:db;create=true';
          • create function leftTable
          • (
          • columnNames varchar( 32672 ),
          • rowContents varchar( 32672 ) ...
          • )
          • returns table
          • (
          • a varchar( 5 ),
          • b varchar( 5 )
          • )
          • language java parameter style derby_jdbc_result_set no sql
          • external name 'org.apache.derbyTesting.functionTests.tests.lang.VarargsRoutines.stringArrayTable';
          • select * from table( leftTable( 'A B', 'APP T', 'APP S' ) ) l;
          • </pre>
            */

          Touches the following file:

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java

          Show
          Rick Hillegas added a comment - Attaching derby-3069-08-aa-stringArrayTable.diff. This patch adds a VTI which I find useful for ad-hoc testing of VTIs. Committed at revision 1431623. Here is the javadoc for the VTI: /** <p> This is a table function which creates a StringArrayVTI out of a space separated list of column names, and a varargs of rows. Each row is a space separated list of column values. Here is a sample usage: </p> * <pre> connect 'jdbc:derby:memory:db;create=true'; create function leftTable ( columnNames varchar( 32672 ), rowContents varchar( 32672 ) ... ) returns table ( a varchar( 5 ), b varchar( 5 ) ) language java parameter style derby_jdbc_result_set no sql external name 'org.apache.derbyTesting.functionTests.tests.lang.VarargsRoutines.stringArrayTable'; select * from table( leftTable( 'A B', 'APP T', 'APP S' ) ) l; </pre> */ Touches the following file: M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java
          Hide
          Rick Hillegas added a comment -

          Attaching a second rev of the functional spec. This clarifies the following:

          o Varargs are also allowed if the parameter style is DERBY_JDBC_RESULT_SET.

          At this point, I think that the feature is ready to be documented and buddy-tested. Thanks.

          Show
          Rick Hillegas added a comment - Attaching a second rev of the functional spec. This clarifies the following: o Varargs are also allowed if the parameter style is DERBY_JDBC_RESULT_SET. At this point, I think that the feature is ready to be documented and buddy-tested. Thanks.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-07-ab-datatypeTests.diff. This patch disables the new tests if you are running on a CDC platform. Tests passed cleanly for me on the previous rev of the patch. Committed at subversion revision 1423714.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-07-ab-datatypeTests.diff. This patch disables the new tests if you are running on a CDC platform. Tests passed cleanly for me on the previous rev of the patch. Committed at subversion revision 1423714.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-07-aa-datatypeTests.diff. This patch adds varargs tests for args of all possible datatypes and for legal coercions to those datatypes. This patch also makes one functional change, making sure that table functions are not created with parameter style DERBY.

          Touches the following files:

          ------------------

          M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj

          Prevent table functions from being created with parameter style DERBY.

          ------------------

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java

          Additional test cases.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-07-aa-datatypeTests.diff. This patch adds varargs tests for args of all possible datatypes and for legal coercions to those datatypes. This patch also makes one functional change, making sure that table functions are not created with parameter style DERBY. Touches the following files: ------------------ M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj Prevent table functions from being created with parameter style DERBY. ------------------ M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java Additional test cases.
          Hide
          Rick Hillegas added a comment -

          Tests passed cleanly for me on derby-3069-06-ab-fixBugs.diff. Committed at subversion revision 1421052.

          Show
          Rick Hillegas added a comment - Tests passed cleanly for me on derby-3069-06-ab-fixBugs.diff. Committed at subversion revision 1421052.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-06-ab-fixBugs.diff. This patch fixes various bugs in the binding and code-generating of varargs calls. I am running regression tests now.

          This patch fixes the following bugs:

          1) Primitive vararg args were being treated as the correponding wrapper types.

          2) OUT and INOUT varargs to procedures needed extra logic to handle the extra array level added by OUT and INOUT args.

          In addition, this patch adds a number of tests.

          Touches the following files:

          -----------------

          M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java
          M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java

          Changes to handle primitive varargs and OUT/INOUT procedure parameters.

          -----------------

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java

          More tests.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-06-ab-fixBugs.diff. This patch fixes various bugs in the binding and code-generating of varargs calls. I am running regression tests now. This patch fixes the following bugs: 1) Primitive vararg args were being treated as the correponding wrapper types. 2) OUT and INOUT varargs to procedures needed extra logic to handle the extra array level added by OUT and INOUT args. In addition, this patch adds a number of tests. Touches the following files: ----------------- M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java Changes to handle primitive varargs and OUT/INOUT procedure parameters. ----------------- M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java More tests.
          Hide
          Rick Hillegas added a comment -

          Committed derby-3069-05-aa-executeVarargs.diff at subversion revision 1420165.

          Show
          Rick Hillegas added a comment - Committed derby-3069-05-aa-executeVarargs.diff at subversion revision 1420165.
          Hide
          Rick Hillegas added a comment -

          Tests passed cleanly for me on derby-3069-05-aa-executeVarargs.diff.

          Show
          Rick Hillegas added a comment - Tests passed cleanly for me on derby-3069-05-aa-executeVarargs.diff.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-05-aa-executeVarargs.diff. This patch wires in bind() and generate() logic for vararg routines so that they can now be invoked. Many more tests need to be written but this patch supplies some initial test cases for invoking vararg routines. I am running regression tests now.

          Touches the following files:

          -----------------

          M java/engine/org/apache/derby/iapi/services/loader/Java5ClassInspector.java
          M java/engine/org/apache/derby/iapi/services/loader/ClassInspector.java

          Added a method to introspect whether a Java method is a varargs method.

          -----------------

          M java/engine/org/apache/derby/impl/sql/catalog/DataDictionaryImpl.java

          Fixed a comment and the declaration of the pi() system method.

          -----------------

          M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java
          M java/engine/org/apache/derby/impl/sql/compile/NewInvocationNode.java
          M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java

          A couple changes:

          1) For a varargs routine, at bind() time coerce the trailing invocation parameters to the declared type of the varargs argument. This involved factoring out the coercion logic into a separate method: coerceMethodParameter().

          2) For a varargs routine, at generate() time stuff the trailing, coerced invocation parameters into an array, which is the actual generated vararg argument. This involved factoring the parameter-generating logic into a separate method: generateAndCastOneParameter().

          I also changed the formatting of the unreadable code I was touching and corrected one of my pet peeves: putting brackets around one-line consequences of "if" statements. This makes the changes to these classes look more substantial.

          -----------------

          A java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java
          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/build.xml
          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java

          More tests.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-05-aa-executeVarargs.diff. This patch wires in bind() and generate() logic for vararg routines so that they can now be invoked. Many more tests need to be written but this patch supplies some initial test cases for invoking vararg routines. I am running regression tests now. Touches the following files: ----------------- M java/engine/org/apache/derby/iapi/services/loader/Java5ClassInspector.java M java/engine/org/apache/derby/iapi/services/loader/ClassInspector.java Added a method to introspect whether a Java method is a varargs method. ----------------- M java/engine/org/apache/derby/impl/sql/catalog/DataDictionaryImpl.java Fixed a comment and the declaration of the pi() system method. ----------------- M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java M java/engine/org/apache/derby/impl/sql/compile/NewInvocationNode.java M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java A couple changes: 1) For a varargs routine, at bind() time coerce the trailing invocation parameters to the declared type of the varargs argument. This involved factoring out the coercion logic into a separate method: coerceMethodParameter(). 2) For a varargs routine, at generate() time stuff the trailing, coerced invocation parameters into an array, which is the actual generated vararg argument. This involved factoring the parameter-generating logic into a separate method: generateAndCastOneParameter(). I also changed the formatting of the unreadable code I was touching and corrected one of my pet peeves: putting brackets around one-line consequences of "if" statements. This makes the changes to these classes look more substantial. ----------------- A java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsRoutines.java M java/testing/org/apache/derbyTesting/functionTests/tests/lang/build.xml M java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java More tests.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-04-aa-shortenRoutineNamesInUpgradeTest.diff. My previous submission raised an error in the upgrade tests. This patch should fix that problem. Committed at subversion revision 1418022.

          The problem was that my new upgrade test created functions and procedures with names which were longer than 18 characters. In 10.0, the very first release of Derby, that's the limit on the length of SQL identifiers. In 10.1, the limit was raised to 128 (see submission 170069 by Sateesh).

          This patch shortens the lengths of the problematic identifiers. With this patch, the upgrade tests run cleanly for me when the initial release is 10.0.2.1.

          Touches the following file:

          M java/testing/org/apache/derbyTesting/functionTests/tests/upgradeTests/Changes10_10.java

          Show
          Rick Hillegas added a comment - Attaching derby-3069-04-aa-shortenRoutineNamesInUpgradeTest.diff. My previous submission raised an error in the upgrade tests. This patch should fix that problem. Committed at subversion revision 1418022. The problem was that my new upgrade test created functions and procedures with names which were longer than 18 characters. In 10.0, the very first release of Derby, that's the limit on the length of SQL identifiers. In 10.1, the limit was raised to 128 (see submission 170069 by Sateesh). This patch shortens the lengths of the problematic identifiers. With this patch, the upgrade tests run cleanly for me when the initial release is 10.0.2.1. Touches the following file: M java/testing/org/apache/derbyTesting/functionTests/tests/upgradeTests/Changes10_10.java
          Hide
          Rick Hillegas added a comment -

          I committed derby-3069-03-ab-varargsSyntax.diff at subversion revision 1417835.

          Show
          Rick Hillegas added a comment - I committed derby-3069-03-ab-varargsSyntax.diff at subversion revision 1417835.
          Hide
          Rick Hillegas added a comment -

          Tests passed cleanly for me on derby-3069-03-ab-varargsSyntax.diff.

          Show
          Rick Hillegas added a comment - Tests passed cleanly for me on derby-3069-03-ab-varargsSyntax.diff.
          Hide
          Rick Hillegas added a comment -

          The current rev of the functional spec needs a correction: The spec says that the varargs ... delimiter is only allowed if the parameter style is DERBY. I need to update to say that varargs are also allowed for parameter style DERBY_JDBC_RESULT_SET. Before posting a new spec, I will wait to see if other corrections to the spec arise during implementation.

          Show
          Rick Hillegas added a comment - The current rev of the functional spec needs a correction: The spec says that the varargs ... delimiter is only allowed if the parameter style is DERBY. I need to update to say that varargs are also allowed for parameter style DERBY_JDBC_RESULT_SET. Before posting a new spec, I will wait to see if other corrections to the spec arise during implementation.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-03-ab-varargsSyntax.diff. I needed to update some more canons for the dblook tests. I will re-run the regression tests.

          Touches the following additional files:

          -------------

          M java/testing/org/apache/derbyTesting/functionTests/master/dblook_test_territory.out
          M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/dblook_test_net.out
          M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/dblook_test_net_territory.out

          Show
          Rick Hillegas added a comment - Attaching derby-3069-03-ab-varargsSyntax.diff. I needed to update some more canons for the dblook tests. I will re-run the regression tests. Touches the following additional files: ------------- M java/testing/org/apache/derbyTesting/functionTests/master/dblook_test_territory.out M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/dblook_test_net.out M java/testing/org/apache/derbyTesting/functionTests/master/DerbyNetClient/dblook_test_net_territory.out
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-03-aa-varargsSyntax.diff. This patch adds DDL support for vararg routines. I am running regression tests now.

          This patch adds support for declaring vararg functions and procedures. Syntax like the following is supported:

          create procedure varargsderbystyle ( a int ... )
          language java parameter style derby no sql external name 'Foo.foo' ;

          create function varargsderbystyle ( a int ... ) returns integer
          language java parameter style derby no sql external name 'Foo.foo' ;

          create function varargstablefunction ( a int ... )
          returns table ( b int )
          language java parameter style derby_jdbc_result_set no sql external name 'Foo.foo' ;

          Invocations of these routines aren't bound to actual varargs methods yet. That will happen in a follow-on patch.

          Touches the following files:

          -------------

          M java/engine/org/apache/derby/catalog/types/RoutineAliasInfo.java

          Metadata changes to support vararg routines and the new DERBY parameter style.

          -------------

          M java/engine/org/apache/derby/impl/sql/catalog/DataDictionaryImpl.java

          Data dictionary changes to declare existing functions/procedures as non-vararg routines.

          -------------

          M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj
          M java/engine/org/apache/derby/impl/sql/compile/CreateAliasNode.java

          Parse- and bind-time support for declaring vararg routines.

          -------------

          M java/engine/org/apache/derby/loc/messages.xml
          M java/shared/org/apache/derby/shared/common/reference/SQLState.java

          New error messages.

          -------------

          M java/testing/org/apache/derbyTesting/functionTests/tests/lang/_Suite.java
          A java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java

          Syntax tests for declaring vararg routines.

          -------------

          M java/testing/org/apache/derbyTesting/functionTests/tests/tools/dblook_makeDB.sql
          M java/testing/org/apache/derbyTesting/functionTests/master/dblook_test.out

          Test to verify that dblook turns out the correct DDL for vararg routines.

          -------------

          M java/testing/org/apache/derbyTesting/functionTests/tests/upgradeTests/Changes10_10.java

          Upgrade tests.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-03-aa-varargsSyntax.diff. This patch adds DDL support for vararg routines. I am running regression tests now. This patch adds support for declaring vararg functions and procedures. Syntax like the following is supported: create procedure varargsderbystyle ( a int ... ) language java parameter style derby no sql external name 'Foo.foo' ; create function varargsderbystyle ( a int ... ) returns integer language java parameter style derby no sql external name 'Foo.foo' ; create function varargstablefunction ( a int ... ) returns table ( b int ) language java parameter style derby_jdbc_result_set no sql external name 'Foo.foo' ; Invocations of these routines aren't bound to actual varargs methods yet. That will happen in a follow-on patch. Touches the following files: ------------- M java/engine/org/apache/derby/catalog/types/RoutineAliasInfo.java Metadata changes to support vararg routines and the new DERBY parameter style. ------------- M java/engine/org/apache/derby/impl/sql/catalog/DataDictionaryImpl.java Data dictionary changes to declare existing functions/procedures as non-vararg routines. ------------- M java/engine/org/apache/derby/impl/sql/compile/sqlgrammar.jj M java/engine/org/apache/derby/impl/sql/compile/CreateAliasNode.java Parse- and bind-time support for declaring vararg routines. ------------- M java/engine/org/apache/derby/loc/messages.xml M java/shared/org/apache/derby/shared/common/reference/SQLState.java New error messages. ------------- M java/testing/org/apache/derbyTesting/functionTests/tests/lang/_Suite.java A java/testing/org/apache/derbyTesting/functionTests/tests/lang/VarargsTest.java Syntax tests for declaring vararg routines. ------------- M java/testing/org/apache/derbyTesting/functionTests/tests/tools/dblook_makeDB.sql M java/testing/org/apache/derbyTesting/functionTests/master/dblook_test.out Test to verify that dblook turns out the correct DDL for vararg routines. ------------- M java/testing/org/apache/derbyTesting/functionTests/tests/upgradeTests/Changes10_10.java Upgrade tests.
          Hide
          Rick Hillegas added a comment -

          Thanks for that feedback, Knut. Attaching Varargs.html, the first draft of a functional spec for this feature.

          Show
          Rick Hillegas added a comment - Thanks for that feedback, Knut. Attaching Varargs.html, the first draft of a functional spec for this feature.
          Hide
          Knut Anders Hatlen added a comment -

          Using a special parameter style to enable this functionality sounds fine to me. That's the same thing we do for table functions, and it would prevent conflicts with future standardization in this area.

          As to item 3 above, I'd suggest that we go for option 3a in the first iteration, and revisit it when we have more experience with how vararg functions are used.

          Show
          Knut Anders Hatlen added a comment - Using a special parameter style to enable this functionality sounds fine to me. That's the same thing we do for table functions, and it would prevent conflicts with future standardization in this area. As to item 3 above, I'd suggest that we go for option 3a in the first iteration, and revisit it when we have more experience with how vararg functions are used.
          Hide
          Rick Hillegas added a comment - - edited

          Here is my current thinking on this topic:

          1) We would clearly mark this as a Derby extension by using a new DERBY parameter style.

          2) We would clearly mark routines which were meant to resolve against vararg methods. If the last (and only the last) argument of the routine declaration ended with an ellipsis token (...), then the routine would have to resolve to a Java varargs method.

          So, for example, consider the following SQL function definition. Note the use of the DERBY parameter style and the ellipsis token:

          create function maximum( a int... ) returns int
          language java parameter style derby no sql
          external name 'IntFunctions.maximum';

          This SQL signature would resolve to the following Java vararg method:

          public static Integer maximum( Integer... args )
          {
          if ( args == null )

          { return null; }
          if ( args.length == 0 ) { return null; }

          int result = Integer.MIN_VALUE;
          for ( Integer value : args )
          {
          if ( value == null )

          { return null; }

          else

          { result = Math.max( result, value.intValue() ); }

          }

          return result;
          }

          3) We would need to decide what to do about procedures which return dynamic ResultSets. That is because dynamic ResultSets and Java varargs both make claims on the trailing arguments of the Java method signature. Here are some possible ways to handle this problem:

          3a) Do not allow varargs if the procedure is defined with a DYNAMIC RESULT SETS greater than 0.

          3b) Allow the Java varargs argument to come AFTER the ResultSet[] arguments. So the Java method signature would begin with the non-vararg arguments, followed by a number of ResultSet[] arguments equal to the value of DYNAMIC RESULT SETS, followed by a trailing ellipsis tagged argument. So this SQL declaration

          create procedure varargProcWithDynamicResultSets
          ( in a int, b int, in f bigint... )
          language java parameter style derby reads sql data
          dynamic result sets 2
          external name 'Procs.varargProcWithDynamicResultSets';

          would bind to this Java signature:

          public static void varargProcWithDynamicResultSets
          ( int a, int b, int c, ResultSet[] d, ResultSet[] e, long... f )

          What do other people think?

          Thanks,
          -Rick

          Show
          Rick Hillegas added a comment - - edited Here is my current thinking on this topic: 1) We would clearly mark this as a Derby extension by using a new DERBY parameter style. 2) We would clearly mark routines which were meant to resolve against vararg methods. If the last (and only the last) argument of the routine declaration ended with an ellipsis token (...), then the routine would have to resolve to a Java varargs method. So, for example, consider the following SQL function definition. Note the use of the DERBY parameter style and the ellipsis token: create function maximum( a int... ) returns int language java parameter style derby no sql external name 'IntFunctions.maximum'; This SQL signature would resolve to the following Java vararg method: public static Integer maximum( Integer... args ) { if ( args == null ) { return null; } if ( args.length == 0 ) { return null; } int result = Integer.MIN_VALUE; for ( Integer value : args ) { if ( value == null ) { return null; } else { result = Math.max( result, value.intValue() ); } } return result; } 3) We would need to decide what to do about procedures which return dynamic ResultSets. That is because dynamic ResultSets and Java varargs both make claims on the trailing arguments of the Java method signature. Here are some possible ways to handle this problem: 3a) Do not allow varargs if the procedure is defined with a DYNAMIC RESULT SETS greater than 0. 3b) Allow the Java varargs argument to come AFTER the ResultSet[] arguments. So the Java method signature would begin with the non-vararg arguments, followed by a number of ResultSet[] arguments equal to the value of DYNAMIC RESULT SETS, followed by a trailing ellipsis tagged argument. So this SQL declaration create procedure varargProcWithDynamicResultSets ( in a int, b int, in f bigint... ) language java parameter style derby reads sql data dynamic result sets 2 external name 'Procs.varargProcWithDynamicResultSets'; would bind to this Java signature: public static void varargProcWithDynamicResultSets ( int a, int b, int c, ResultSet[] d, ResultSet[] e, long... f ) What do other people think? Thanks, -Rick
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-02-backout.diff, which backs out this feature. I do not have time to polish this before 10.4 code freeze. Committed at subversion revision 630351.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-02-backout.diff, which backs out this feature. I do not have time to polish this before 10.4 code freeze. Committed at subversion revision 630351.
          Hide
          Dyre Tjeldvoll added a comment -

          Since the discussion appears to have stalled (last comment is from 5 Oct) before reaching a consensus, I'm removing the 'patch available' flag.

          Show
          Dyre Tjeldvoll added a comment - Since the discussion appears to have stalled (last comment is from 5 Oct) before reaching a consensus, I'm removing the 'patch available' flag.
          Hide
          Rick Hillegas added a comment -

          Thanks for the continued discussion, Dan. Some more responses follow:

          A) If we need another parameter style, I think that's ok. I would just like to keep the number of parameter styles to a minimum.

          B) To me there is a lot of value in taking advantage of Java 5 varargs even though you can't declare vararg SQL signatures yet. I agree that it is awkward that your application code still has to declare a separate SQL method for each new signature that it needs. Without support for Java 5 varargs, the application has to do the following additional work too:

          i) Generate and compile Java wrapper code for the new signature.

          ii) Replace the relevant server jar file either directly on the classpath or as stored in the database.

          I find that eliminating these steps is a welcome benefit.

          2) I can't say whether other people would find the INOUT/OUT resolution confusing. It seems straightforward to me.

          3) I haven't found a need for ResultSet varargs yet so I don't have any experience to suggest whether this makes the code easier or harder to write. However, I think that the advantages described for (B) still apply here. If we continue with the behavior described for (1), then I think that the existing SQL Standard rules continue to apply.

          Because I haven't personally needed ResultSet varargs yet, I don't have a lot of passion about the issue. We could not support ResultSet varargs for 10.4. If someone needed this feature later on, they could add it--by adding a third phase to the resolution process, we could continue to preserve backward compatibility.

          Show
          Rick Hillegas added a comment - Thanks for the continued discussion, Dan. Some more responses follow: A) If we need another parameter style, I think that's ok. I would just like to keep the number of parameter styles to a minimum. B) To me there is a lot of value in taking advantage of Java 5 varargs even though you can't declare vararg SQL signatures yet. I agree that it is awkward that your application code still has to declare a separate SQL method for each new signature that it needs. Without support for Java 5 varargs, the application has to do the following additional work too: i) Generate and compile Java wrapper code for the new signature. ii) Replace the relevant server jar file either directly on the classpath or as stored in the database. I find that eliminating these steps is a welcome benefit. 2) I can't say whether other people would find the INOUT/OUT resolution confusing. It seems straightforward to me. 3) I haven't found a need for ResultSet varargs yet so I don't have any experience to suggest whether this makes the code easier or harder to write. However, I think that the advantages described for (B) still apply here. If we continue with the behavior described for (1), then I think that the existing SQL Standard rules continue to apply. Because I haven't personally needed ResultSet varargs yet, I don't have a lot of passion about the issue. We could not support ResultSet varargs for 10.4. If someone needed this feature later on, they could add it--by adding a third phase to the resolution process, we could continue to preserve backward compatibility.
          Hide
          Daniel John Debrunner added a comment -

          Thanks Rick, I'm still not sure that varargs should be resolved to when using PARAMETER STYLE JAVA for two reasons, but it's better to discuss that with a concrete detailed proposal, which we are heading towards. The two reasons I see for not supporting it are:

          A) It's non standard, an application using this would fail to run on a database following the standard, thus breaking Derby's charter.

          B) It doesn't provide much benefit, it's forcing a varargs method on a calling sequence that does not have variable numbers of arguments. SQL routines have fixed number of arguments. An application can always provide a wrapper method for a SQL routine that calls the varargs method.

          Having said that, here's some feedback on the latest comment.

          1) looks reasonable.

          2) What you describe does not match how varargs work in Java today. Not saying that's good or bad, just pointing it out.

          v1(new String[1]) will resolve to a method v1(String ...)

          Will it be confusing if Derby's resolution to varargs methods does not match Java semantics?

          I also wonder if there's a bug (before any of your changes) where an INOUT/OUT parameter would incorrectly resolve to a varargs method.

          3) Maybe it's just the non-standard nature that makes me think DYNAMIC RESULT SETS should not resolve to varargs. I can an argument that allowing resolution to varargs methods that form part of some external library, like J2SE libraries, JavaMail etc. But no library is going to have method that is suitable for dynamic result sets, thus any method for dynamic result sets is going to be specific for that application, so what's the benefit of varargs?

          Within the implementation of such a method, does the use of varargs make the code easier to understand?

          customerList[0] = ps.executeQuery(); // non-varargs allows the variables holding the ResultSet to be specifically named

          drs[2][0] = ps.executeQuery(); // varargs, single name for the variable used to return result sets (or additional code to create local variables)

          One other issue here is how big an array should Derby pass into the varargs method? Is it as simple as the dynamic result set count? I don't think that's how the non-varargs method works, I think it can resolve to a method (as per spec) with any number of ResultSet[] parameters. Ie. with a maximum of three dynamic result sets the method may have holders for six ResultSets, but only return three, or could return five with two closed.

          DYNAMIC RESULT SETS is a variable number of ResultSets on the return from the method, not the input to the method, thus I don't think it really fits the Java varargs paradigm.

          Show
          Daniel John Debrunner added a comment - Thanks Rick, I'm still not sure that varargs should be resolved to when using PARAMETER STYLE JAVA for two reasons, but it's better to discuss that with a concrete detailed proposal, which we are heading towards. The two reasons I see for not supporting it are: A) It's non standard, an application using this would fail to run on a database following the standard, thus breaking Derby's charter. B) It doesn't provide much benefit, it's forcing a varargs method on a calling sequence that does not have variable numbers of arguments. SQL routines have fixed number of arguments. An application can always provide a wrapper method for a SQL routine that calls the varargs method. Having said that, here's some feedback on the latest comment. 1) looks reasonable. 2) What you describe does not match how varargs work in Java today. Not saying that's good or bad, just pointing it out. v1(new String [1] ) will resolve to a method v1(String ...) Will it be confusing if Derby's resolution to varargs methods does not match Java semantics? I also wonder if there's a bug (before any of your changes) where an INOUT/OUT parameter would incorrectly resolve to a varargs method. 3) Maybe it's just the non-standard nature that makes me think DYNAMIC RESULT SETS should not resolve to varargs. I can an argument that allowing resolution to varargs methods that form part of some external library, like J2SE libraries, JavaMail etc. But no library is going to have method that is suitable for dynamic result sets, thus any method for dynamic result sets is going to be specific for that application, so what's the benefit of varargs? Within the implementation of such a method, does the use of varargs make the code easier to understand? customerList [0] = ps.executeQuery(); // non-varargs allows the variables holding the ResultSet to be specifically named drs [2] [0] = ps.executeQuery(); // varargs, single name for the variable used to return result sets (or additional code to create local variables) One other issue here is how big an array should Derby pass into the varargs method? Is it as simple as the dynamic result set count? I don't think that's how the non-varargs method works, I think it can resolve to a method (as per spec) with any number of ResultSet[] parameters. Ie. with a maximum of three dynamic result sets the method may have holders for six ResultSets, but only return three, or could return five with two closed. DYNAMIC RESULT SETS is a variable number of ResultSets on the return from the method, not the input to the method, thus I don't think it really fits the Java varargs paradigm.
          Hide
          Rick Hillegas added a comment -

          Thanks again for helping me puzzle through this, Dan. Here are some responses to your questions:

          1) I like what you propose: if there is a non-varargs resolution for a routine invocation, then that should be preferred over a varargs resolution. I think that this means that we would keep two lists of candidate methods as we resolve the invocation: JCS (already described in section 8.6) and JCVS (the set of candidate resolutions which have vararg signatures). The extra varargs methods which I proposed to put into JCS would instead go into JCVS. Then we would continue:

          a) If JCS has exactly one method in it, that is the resolution of the routine invocation.

          b) If JCS has more than one method in it, then we raise an error.

          c) Otherwise JCS is empty and we continue:

          d) If JCVS has exactly one method in it, then that is the resolution of the routine invocation.

          e) Otherwise, we raise an error.

          Does this sound reasonable?

          2) I think that INOUT and OUT parameters can resolve to varargs provided that we are careful that the vararg type is itself an array. So, for instance if we declare

          create procedure foo( inout arg1 varchar( 30 ) ) ...

          then an invocation of this procedure could resolve to the following method

          public static void fooRes( String[]... inoutargs )

          but not

          public static void fooRes( String... args)

          Right now, we are not correcty resolving these INOUT/OUT parameters against varargs signatures. That is, we are not detecting the vararg candidates and so resolution fails. That's a bug I need to fix. Are you seeing other subtleties here?

          3) I don't think I understand the subtleties of DYNAMIC RESULT SETS. Could you elaborate on why you think that we should not support these as varargs? Off the top of my head, it seems that we ought to be able to resolve dynamic result sets to varargs. So for instance, if we declare the following procedure:

          create procedure dynamicResultSet( arg1 int ) ... dynamic result sets 2 ...

          then an invocation of this procedure should be resolvable to

          public static void dynamicResultSetRes( int arg1, ResultSet[]... dynamicResultSets )

          4) I agree that it is unfortunate that we can't mix vararg parameters with dynamic result sets.

          Show
          Rick Hillegas added a comment - Thanks again for helping me puzzle through this, Dan. Here are some responses to your questions: 1) I like what you propose: if there is a non-varargs resolution for a routine invocation, then that should be preferred over a varargs resolution. I think that this means that we would keep two lists of candidate methods as we resolve the invocation: JCS (already described in section 8.6) and JCVS (the set of candidate resolutions which have vararg signatures). The extra varargs methods which I proposed to put into JCS would instead go into JCVS. Then we would continue: a) If JCS has exactly one method in it, that is the resolution of the routine invocation. b) If JCS has more than one method in it, then we raise an error. c) Otherwise JCS is empty and we continue: d) If JCVS has exactly one method in it, then that is the resolution of the routine invocation. e) Otherwise, we raise an error. Does this sound reasonable? 2) I think that INOUT and OUT parameters can resolve to varargs provided that we are careful that the vararg type is itself an array. So, for instance if we declare create procedure foo( inout arg1 varchar( 30 ) ) ... then an invocation of this procedure could resolve to the following method public static void fooRes( String[]... inoutargs ) but not public static void fooRes( String... args) Right now, we are not correcty resolving these INOUT/OUT parameters against varargs signatures. That is, we are not detecting the vararg candidates and so resolution fails. That's a bug I need to fix. Are you seeing other subtleties here? 3) I don't think I understand the subtleties of DYNAMIC RESULT SETS. Could you elaborate on why you think that we should not support these as varargs? Off the top of my head, it seems that we ought to be able to resolve dynamic result sets to varargs. So for instance, if we declare the following procedure: create procedure dynamicResultSet( arg1 int ) ... dynamic result sets 2 ... then an invocation of this procedure should be resolvable to public static void dynamicResultSetRes( int arg1, ResultSet[]... dynamicResultSets ) 4) I agree that it is unfortunate that we can't mix vararg parameters with dynamic result sets.
          Hide
          Rick Hillegas added a comment -

          Thanks for the quick response, Dan. Unfortunately, I'm going to be on the road for the next couple days and won't have time to give you a thoughtful response until the end of the week. Please bear with me.

          Show
          Rick Hillegas added a comment - Thanks for the quick response, Dan. Unfortunately, I'm going to be on the road for the next couple days and won't have time to give you a thoughtful response until the end of the week. Please bear with me.
          Hide
          Daniel John Debrunner added a comment -

          Thanks Rick, some questions based upon this:

          1) When Java resolves to methods with varargs it's a two phase process, so that if a non-varargs method that matches the arguments exists, it takes precedence over any varargs methods that matches the arguments. It seems with what you are describing this paradigm is not followed, which means that the presence of a varargs method could break an existing application once this improvement is enabled. One extreme case of this is the method foo(T ...) for any type T will cause failure when the Java routine with no parameters previously mapped to foo().

          Is this potential to break existing applications intentional, or should the method resolution follow something similar to Java's two stage process?

          http://java.sun.com/docs/books/jls/third_edition/html/expressions.html#15.12

          2) Any issues with INOUT or OUT parameters that end up being mapped to the varargs? Especially for how the implementation of the varargs method would access the INOUT/OUT parameters?

          3) How about varargs with DYNAMIC RESULT SETS, can the return of the ResultSet's be through a varargs parameter? I assume not.

          4) Java's limitation to a single varargs argument in the last position leads to a somewhat strange situation. A procedure with multiple SQL parameters of the same type can map to a varargs method only when DYNAMIC RESULT SETS is zero.

          Show
          Daniel John Debrunner added a comment - Thanks Rick, some questions based upon this: 1) When Java resolves to methods with varargs it's a two phase process, so that if a non-varargs method that matches the arguments exists, it takes precedence over any varargs methods that matches the arguments. It seems with what you are describing this paradigm is not followed, which means that the presence of a varargs method could break an existing application once this improvement is enabled. One extreme case of this is the method foo(T ...) for any type T will cause failure when the Java routine with no parameters previously mapped to foo(). Is this potential to break existing applications intentional, or should the method resolution follow something similar to Java's two stage process? http://java.sun.com/docs/books/jls/third_edition/html/expressions.html#15.12 2) Any issues with INOUT or OUT parameters that end up being mapped to the varargs? Especially for how the implementation of the varargs method would access the INOUT/OUT parameters? 3) How about varargs with DYNAMIC RESULT SETS, can the return of the ResultSet's be through a varargs parameter? I assume not. 4) Java's limitation to a single varargs argument in the last position leads to a somewhat strange situation. A procedure with multiple SQL parameters of the same type can map to a varargs method only when DYNAMIC RESULT SETS is zero.
          Hide
          Rick Hillegas added a comment -

          Here's a stab at a functional spec for this behavior:

          User Experience:

          1) Functions and Procedures (with fixed numbers of arguments) can resolve to Java methods with varargs signatures when Derby runs on Java 5 or higher.

          2) Does NOT allow varargs in the SQL declaration of the arguments of Functions and Procedures.

          Method Resolution Details:

          The method resolution rules for SQL invoked routines are described in the SQL Standard, part 13, section 8.6. We change the resolution rules given in Rule 3, case b, step 4, case 1. More specifically, we expand JCS to include additional candidate methods. The additional methods obey the following rules.

          Let the number of arguments in JPDL be SQLArgCount. Let Candidate be a method which satisfies the following:

          aa) Candidate is a visible Java method in class JCLSN in Jar JN with method name JMN.

          bb) Candidate has CandidateArgCount arguments in its signature. The last argument in that signature is a vararg argument of type T.

          cc) The first CandidateArgCount-1 arguments of JPDL have the types of the first CandidateArgCount-1 arguments of Candidate.

          dd) If SQLArgCount == CandidateArgCount-1, then Candidate is added to JCS.

          ee) If SQLArgCount >= CandidateArgCount and all of the arguments in JPDL from CandidateArgCount onward have type T, then Candidate is added to JCS.

          Show
          Rick Hillegas added a comment - Here's a stab at a functional spec for this behavior: User Experience: 1) Functions and Procedures (with fixed numbers of arguments) can resolve to Java methods with varargs signatures when Derby runs on Java 5 or higher. 2) Does NOT allow varargs in the SQL declaration of the arguments of Functions and Procedures. Method Resolution Details: The method resolution rules for SQL invoked routines are described in the SQL Standard, part 13, section 8.6. We change the resolution rules given in Rule 3, case b, step 4, case 1. More specifically, we expand JCS to include additional candidate methods. The additional methods obey the following rules. Let the number of arguments in JPDL be SQLArgCount. Let Candidate be a method which satisfies the following: aa) Candidate is a visible Java method in class JCLSN in Jar JN with method name JMN. bb) Candidate has CandidateArgCount arguments in its signature. The last argument in that signature is a vararg argument of type T. cc) The first CandidateArgCount-1 arguments of JPDL have the types of the first CandidateArgCount-1 arguments of Candidate. dd) If SQLArgCount == CandidateArgCount-1, then Candidate is added to JCS. ee) If SQLArgCount >= CandidateArgCount and all of the arguments in JPDL from CandidateArgCount onward have type T, then Candidate is added to JCS.
          Hide
          Daniel John Debrunner added a comment -

          > I do not believe that varargs change the definition of JPDL.
          > However, I believe that varargs can add more candidate methods to JCS.
          > So I believe it is that delta to JCS which must be described. Again, does that sound correct to you?

          I think this all depends on what you intend to implement (or have implemented). E.g. JPDL could support a varargs declaration in its <Java parameter declaration list>, not sure if you intend to do this or not, since it's non standard I can't guess what you are trying to implement, thus it's hard to review any patch as it's unclear what the intended behave is, beyond a vague "support varargs" statement.

          Show
          Daniel John Debrunner added a comment - > I do not believe that varargs change the definition of JPDL. > However, I believe that varargs can add more candidate methods to JCS. > So I believe it is that delta to JCS which must be described. Again, does that sound correct to you? I think this all depends on what you intend to implement (or have implemented). E.g. JPDL could support a varargs declaration in its <Java parameter declaration list>, not sure if you intend to do this or not, since it's non standard I can't guess what you are trying to implement, thus it's hard to review any patch as it's unclear what the intended behave is, beyond a vague "support varargs" statement.
          Hide
          Rick Hillegas added a comment -

          Let's figure out what a functional spec for this would look like. I think that we have to start out assuming that Derby implements the relevant rules in Section 8.6. Given that, I think that a functional spec would consist of describing the delta from those rules needed to support varargs. Does that sound useful and adequate to you?

          In describing a delta, I think that the crucial terms in 8.6 are JPDL and JCS. JPDL is the Java Parameter Datatype List, determined by applying earlier rules to the SQL datatypes of the routine invocation. JCS is the set of candidate methods which the rules consider as possible resolutions of the routine invocation. At the end of the process, there should be exactly 1 method in JCS. I do not believe that varargs change the definition of JPDL. However, I believe that varargs can add more candidate methods to JCS. So I believe it is that delta to JCS which must be described. Again, does that sound correct to you?

          It is easy for me to get lost in section 8.6. However, I think that the relevant part of that section is Rule 3, case b, step 4, case 1. I think that is the subsection which describes how to construct JCS for Derby's SQL invoked routines. Do you agree?

          Show
          Rick Hillegas added a comment - Let's figure out what a functional spec for this would look like. I think that we have to start out assuming that Derby implements the relevant rules in Section 8.6. Given that, I think that a functional spec would consist of describing the delta from those rules needed to support varargs. Does that sound useful and adequate to you? In describing a delta, I think that the crucial terms in 8.6 are JPDL and JCS. JPDL is the Java Parameter Datatype List, determined by applying earlier rules to the SQL datatypes of the routine invocation. JCS is the set of candidate methods which the rules consider as possible resolutions of the routine invocation. At the end of the process, there should be exactly 1 method in JCS. I do not believe that varargs change the definition of JPDL. However, I believe that varargs can add more candidate methods to JCS. So I believe it is that delta to JCS which must be described. Again, does that sound correct to you? It is easy for me to get lost in section 8.6. However, I think that the relevant part of that section is Rule 3, case b, step 4, case 1. I think that is the subsection which describes how to construct JCS for Derby's SQL invoked routines. Do you agree?
          Hide
          Daniel John Debrunner added a comment -

          >> 2) If the Derby engine is running on Java 5 or later, then the
          >> resolution rules are the OldRules with the following change: The SQL
          >> Standard, part 13, section 8.6, refers to the resolution rules
          >> described in The Java Language Specification, Second Edition. Replace
          >> that with a reference to the resolution rules in The Java Language
          >> Specification, Third Edition.

          > I think we should allow ourselves to regard the standard's intent
          > rather than letter here: JAVA PARAMETER STYLE should be allowed to
          > resolve to varargs when running on Java 5 or later, since this is a
          > now a part of Java. (As well as for DERBY_JDBC_RESULT_SET).

          I think we need to be very careful here. Section 8.6 SR 3b case 2) B II is the section that mentions the java language specification and it does not apply to Derby's Java routines (SQL invoked routines).

          SQL resolution of SQL invoked routines to Java static methods does not not follow Java rules, thus one cannot rely on that to justify that varargs can be resolved to because Java supports it.

          It would be really helpful if a functional spec was written indicating when & how varargs resolution will be used and when it will not.

          Show
          Daniel John Debrunner added a comment - >> 2) If the Derby engine is running on Java 5 or later, then the >> resolution rules are the OldRules with the following change: The SQL >> Standard, part 13, section 8.6, refers to the resolution rules >> described in The Java Language Specification, Second Edition. Replace >> that with a reference to the resolution rules in The Java Language >> Specification, Third Edition. > I think we should allow ourselves to regard the standard's intent > rather than letter here: JAVA PARAMETER STYLE should be allowed to > resolve to varargs when running on Java 5 or later, since this is a > now a part of Java. (As well as for DERBY_JDBC_RESULT_SET). I think we need to be very careful here. Section 8.6 SR 3b case 2) B II is the section that mentions the java language specification and it does not apply to Derby's Java routines (SQL invoked routines). SQL resolution of SQL invoked routines to Java static methods does not not follow Java rules, thus one cannot rely on that to justify that varargs can be resolved to because Java supports it. It would be really helpful if a functional spec was written indicating when & how varargs resolution will be used and when it will not.
          Hide
          Rick Hillegas added a comment -

          No strong objections have surfaced. I have committed derby-3069-01-varargs-ab.diff at subversion revision 579653.

          Show
          Rick Hillegas added a comment - No strong objections have surfaced. I have committed derby-3069-01-varargs-ab.diff at subversion revision 579653.
          Hide
          Rick Hillegas added a comment -

          I agree with Dag here. I think that the divergence from the ANSI spec is in line with the intention of the spec. I am inclined to do the following:

          • Commit derby-3069-01-varargs-ab.diff.
          • Add a note to the PARAMETER STYLE sub-topic of the CREATE FUNCTION topic in the Derby Reference Guide. The note would explain the divergence from the ANSI spec.

          Please let me know if there are any strong objections to this approach.

          Show
          Rick Hillegas added a comment - I agree with Dag here. I think that the divergence from the ANSI spec is in line with the intention of the spec. I am inclined to do the following: Commit derby-3069-01-varargs-ab.diff. Add a note to the PARAMETER STYLE sub-topic of the CREATE FUNCTION topic in the Derby Reference Guide. The note would explain the divergence from the ANSI spec. Please let me know if there are any strong objections to this approach.
          Hide
          Rick Hillegas added a comment -

          Attaching a second rev of this patch: derby-3069-01-varargs-ab.diff. This fixes a corner case in the previous rev: In Java 5, a no argument invocation of a method can end up bound to a varargs overload. So, for instance, if you declare

          public static String sampleVarargs( String... args )

          then, barring an ambiguity, the following invocation will resolve to it:

          stringResult = sampleVarargs();

          This patch lets you bind no-arg functions/procedures to vararg static public methods.

          Show
          Rick Hillegas added a comment - Attaching a second rev of this patch: derby-3069-01-varargs-ab.diff. This fixes a corner case in the previous rev: In Java 5, a no argument invocation of a method can end up bound to a varargs overload. So, for instance, if you declare public static String sampleVarargs( String... args ) then, barring an ambiguity, the following invocation will resolve to it: stringResult = sampleVarargs(); This patch lets you bind no-arg functions/procedures to vararg static public methods.
          Hide
          Dag H. Wanvik added a comment -

          > 2) If the Derby engine is running on Java 5 or later, then the
          > resolution rules are the OldRules with the following change: The SQL
          > Standard, part 13, section 8.6, refers to the resolution rules
          > described in The Java Language Specification, Second Edition. Replace
          > that with a reference to the resolution rules in The Java Language
          > Specification, Third Edition.

          I think we should allow ourselves to regard the standard's intent
          rather than letter here: JAVA PARAMETER STYLE should be allowed to
          resolve to varargs when running on Java 5 or later, since this is a
          now a part of Java. (As well as for DERBY_JDBC_RESULT_SET).

          Show
          Dag H. Wanvik added a comment - > 2) If the Derby engine is running on Java 5 or later, then the > resolution rules are the OldRules with the following change: The SQL > Standard, part 13, section 8.6, refers to the resolution rules > described in The Java Language Specification, Second Edition. Replace > that with a reference to the resolution rules in The Java Language > Specification, Third Edition. I think we should allow ourselves to regard the standard's intent rather than letter here: JAVA PARAMETER STYLE should be allowed to resolve to varargs when running on Java 5 or later, since this is a now a part of Java. (As well as for DERBY_JDBC_RESULT_SET).
          Hide
          Rick Hillegas added a comment -

          Dan posted the following questions on derby-dev because JIRA was down:

          >Is there going to be some write up on exactly how the proposed method resolution will work with varargs, so that people can review it and see if it conflicts with the standard method resolution?

          The current resolution rules are mentioned in the Reference Guide section on CREATE FUNCTION in the subsection on PARAMETER STYLE. There the reader is told that the resolution rules are the ones described in part 13 of the SQL Standard. Let us call those resolution rules the OldRules.

          1) If the Derby engine is running on a pre-Java-5 vm (that is, JDK 1.4 or the CDC small device platform), then the resolution rules are the OldRules.

          2) If the Derby engine is running on Java 5 or later, then the resolution rules are the OldRules with the following change: The SQL Standard, part 13, section 8.6, refers to the resolution rules described in The Java Language Specification, Second Edition. Replace that with a reference to the resolution rules in The Java Language Specification, Third Edition.

          >Will this be a different PARAMETER STYLE (than JAVA)? If not what does it say about Derby's commitment to standards when a SQL routine resolves to a method in Derby that it would not resolve to on another database that is following the SQL standard?

          I would have no problem limiting the application of these rules to functions/procedures which are declared with some other PARAMETER STYLE. Currently, we have 2 parameter styles, namely, JAVA and DERBY_JDBC_RESULT_SET. The second style is used for Derby-style table functions as described in DERBY-716. You are only allowed to specify one parameter style according to the SQL Standard, part 2, section 11.50, syntax rule 6a. Now we need to consider the problem of ORing parameter styles together, so that, for instance, we can have a table function resolve against a method with a varags signature. I see a couple solutions to this problem:

          a) Introduce one new parameter style. Let's call it DERBY. Get rid of DERBY_JDBC_RESULT_SET. If the parameter style is DERBY, then table functions resolve to ResultSet-returning-methods and vararg resolution happens according to rule (2) above. In this scheme, there are 2 parameter styles: JAVA and DERBY.

          b) Introduce one new parameter style. Let's call it DERBY_VARARGS. The existing DERBY_JDBC_RESULT_SET parameter style continues to behave as it does today, that is, it resolves table functions to ResultSet-returning-methods but it uses OldRules regardless of the platform. Parameter style DERBY_VARARGS behaves as parameter style DERBY does in case (a). In this scheme, there are 3 parameter styles: JAVA, DERBY_VARARGS, and DERBY_JDBC_RESULT_SET.

          c) Introduce two new parameter styles. Let's call them DERBY_VARARGS_PLAIN and DERBY_VARARGS_JDBC_RESULT_SET. DERBY_VARARGS_PLAIN follows vararg resolution rule (2) but does not resolve table functions to ResultSet-returning-methods. DERBY_VARARGS_JDBC_RESULT_SET behaves as parameter style DERBY does in case (a). In this scheme, there are 4 parameter styles: JAVA, DERBY_JDBC_RESULT_SET, DERBY_VARARGS_PLAIN, and DERBY_VARARGS_JDBC_RESULT_SET.

          I would vote for option (a). As time goes on, people may need to implement other Derby-specific method resolution rules. At that time, they should consider whether to enrich the DERBY style or create a new one.

          Show
          Rick Hillegas added a comment - Dan posted the following questions on derby-dev because JIRA was down: >Is there going to be some write up on exactly how the proposed method resolution will work with varargs, so that people can review it and see if it conflicts with the standard method resolution? The current resolution rules are mentioned in the Reference Guide section on CREATE FUNCTION in the subsection on PARAMETER STYLE. There the reader is told that the resolution rules are the ones described in part 13 of the SQL Standard. Let us call those resolution rules the OldRules. 1) If the Derby engine is running on a pre-Java-5 vm (that is, JDK 1.4 or the CDC small device platform), then the resolution rules are the OldRules. 2) If the Derby engine is running on Java 5 or later, then the resolution rules are the OldRules with the following change: The SQL Standard, part 13, section 8.6, refers to the resolution rules described in The Java Language Specification, Second Edition. Replace that with a reference to the resolution rules in The Java Language Specification, Third Edition. >Will this be a different PARAMETER STYLE (than JAVA)? If not what does it say about Derby's commitment to standards when a SQL routine resolves to a method in Derby that it would not resolve to on another database that is following the SQL standard? I would have no problem limiting the application of these rules to functions/procedures which are declared with some other PARAMETER STYLE. Currently, we have 2 parameter styles, namely, JAVA and DERBY_JDBC_RESULT_SET. The second style is used for Derby-style table functions as described in DERBY-716 . You are only allowed to specify one parameter style according to the SQL Standard, part 2, section 11.50, syntax rule 6a. Now we need to consider the problem of ORing parameter styles together, so that, for instance, we can have a table function resolve against a method with a varags signature. I see a couple solutions to this problem: a) Introduce one new parameter style. Let's call it DERBY. Get rid of DERBY_JDBC_RESULT_SET. If the parameter style is DERBY, then table functions resolve to ResultSet-returning-methods and vararg resolution happens according to rule (2) above. In this scheme, there are 2 parameter styles: JAVA and DERBY. b) Introduce one new parameter style. Let's call it DERBY_VARARGS. The existing DERBY_JDBC_RESULT_SET parameter style continues to behave as it does today, that is, it resolves table functions to ResultSet-returning-methods but it uses OldRules regardless of the platform. Parameter style DERBY_VARARGS behaves as parameter style DERBY does in case (a). In this scheme, there are 3 parameter styles: JAVA, DERBY_VARARGS, and DERBY_JDBC_RESULT_SET. c) Introduce two new parameter styles. Let's call them DERBY_VARARGS_PLAIN and DERBY_VARARGS_JDBC_RESULT_SET. DERBY_VARARGS_PLAIN follows vararg resolution rule (2) but does not resolve table functions to ResultSet-returning-methods. DERBY_VARARGS_JDBC_RESULT_SET behaves as parameter style DERBY does in case (a). In this scheme, there are 4 parameter styles: JAVA, DERBY_JDBC_RESULT_SET, DERBY_VARARGS_PLAIN, and DERBY_VARARGS_JDBC_RESULT_SET. I would vote for option (a). As time goes on, people may need to implement other Derby-specific method resolution rules. At that time, they should consider whether to enrich the DERBY style or create a new one.
          Hide
          Rick Hillegas added a comment -

          This patch passes the existing regression tests. Does not, however, introduce any new tests yet.

          Show
          Rick Hillegas added a comment - This patch passes the existing regression tests. Does not, however, introduce any new tests yet.
          Hide
          Rick Hillegas added a comment -

          Attaching derby-3069-01-aa.diff. This allows you to run functions which are bound to methods which have vararg signatures. Touches the following files:

          M java/engine/org/apache/derby/iapi/services/loader/ClassInspector.java

          This is the compile-time bind() logic. This allows you to resolve function declarations against methods which have vararg signatures. This also exposes helper methods for determining whether a method has a vararg signature.

          M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java
          M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java

          This is the compile-time generate() piece, which generates the byte code needed to invoke varags methods. For varargs signatures, this method builds a concluding array argument and stuffs it with the trailing arguments specified by the user.

          Show
          Rick Hillegas added a comment - Attaching derby-3069-01-aa.diff. This allows you to run functions which are bound to methods which have vararg signatures. Touches the following files: M java/engine/org/apache/derby/iapi/services/loader/ClassInspector.java This is the compile-time bind() logic. This allows you to resolve function declarations against methods which have vararg signatures. This also exposes helper methods for determining whether a method has a vararg signature. M java/engine/org/apache/derby/impl/sql/compile/MethodCallNode.java M java/engine/org/apache/derby/impl/sql/compile/StaticMethodCallNode.java This is the compile-time generate() piece, which generates the byte code needed to invoke varags methods. For varargs signatures, this method builds a concluding array argument and stuffs it with the trailing arguments specified by the user.
          Hide
          Daniel John Debrunner added a comment -

          For section 4.2.2 (second to last paragraph, the one starting "Replace the 4th paragraph") - Derby does not support SQL-invoked methods.

          The PARAMETER STYLE section of CREATE FUNCTION and PROCEDURE state how the method is to be resolved, which is vague in terms of where to find the information but precise in that the rules are defined elsewhere.

          Missed that isVarArgs method, was instead looking for information at the parameter level, thanks

          Show
          Daniel John Debrunner added a comment - For section 4.2.2 (second to last paragraph, the one starting "Replace the 4th paragraph") - Derby does not support SQL-invoked methods. The PARAMETER STYLE section of CREATE FUNCTION and PROCEDURE state how the method is to be resolved, which is vague in terms of where to find the information but precise in that the rules are defined elsewhere. Missed that isVarArgs method, was instead looking for information at the parameter level, thanks
          Hide
          Rick Hillegas added a comment -

          Thanks for the reference to The Java Language Specification, Second Edition late in Part 13, Section 8.6. That section seems to indicate that the Java method resolution rules are to be applied to isolate a set of candidate methods after the types of the arguments have been mapped from SQL types to Java types. The sentence in section 4.2.2 which leapt out at me was this: "If the instance SQL-invoked method is an external Java routine, the term "set of overriding methods" is not applicable; for such methods, the capabilities provided by overriding methods duplicate Java's own mechanisms and the subject routine executed is the one that would be invoked when no overriding methods are specified."

          The CREATE FUNCTION section is vague about how Derby will resolve methods at run-time. Where is this documented?

          The varargs information must be propagated to the classfile. It can be retrieved via reflection by calling Method.isVarArgs().

          Show
          Rick Hillegas added a comment - Thanks for the reference to The Java Language Specification, Second Edition late in Part 13, Section 8.6. That section seems to indicate that the Java method resolution rules are to be applied to isolate a set of candidate methods after the types of the arguments have been mapped from SQL types to Java types. The sentence in section 4.2.2 which leapt out at me was this: "If the instance SQL-invoked method is an external Java routine, the term "set of overriding methods" is not applicable; for such methods, the capabilities provided by overriding methods duplicate Java's own mechanisms and the subject routine executed is the one that would be invoked when no overriding methods are specified." The CREATE FUNCTION section is vague about how Derby will resolve methods at run-time. Where is this documented? The varargs information must be propagated to the classfile. It can be retrieved via reflection by calling Method.isVarArgs().
          Hide
          Daniel John Debrunner added a comment -

          Question though, is the varargs just syntax at the Java language level or is the information propagated all the way to the class file?

          If it's just at the java compiler level then Derby could not implement it as it would be impossible to distinguish between a varargs method and one declared with the corresponding array type. x(String ... args) and x(String[] args) [not even sure how Java handles this]

          From a very quick look at java.lang.reflect I couldn't see any mechanism to determine if a method to a parameter was a varargs one, though I could have missed it.

          Show
          Daniel John Debrunner added a comment - Question though, is the varargs just syntax at the Java language level or is the information propagated all the way to the class file? If it's just at the java compiler level then Derby could not implement it as it would be impossible to distinguish between a varargs method and one declared with the corresponding array type. x(String ... args) and x(String[] args) [not even sure how Java handles this] From a very quick look at java.lang.reflect I couldn't see any mechanism to determine if a method to a parameter was a varargs one, though I could have missed it.
          Hide
          Daniel John Debrunner added a comment -

          In what way is the description of CREATE FUNCTION misleading? I agree that it's vague, it just points to the standards without really explicitly stating which standard, but I don't see anything misleading.

          Show
          Daniel John Debrunner added a comment - In what way is the description of CREATE FUNCTION misleading? I agree that it's vague, it just points to the standards without really explicitly stating which standard, but I don't see anything misleading.
          Hide
          Daniel John Debrunner added a comment -

          Section 8.6 of part 13 is very precise on the method resolution. It even mentions "The Java Language Specification, Second Edition" in one section.

          Can you be more explicit on what you see in the "last paragraph of section 4.2.2" that leads you to believe that Java method resolution is followed?

          Note that Java method resolution is not followed by the SQL Standard, e.g. a VARCHAR() column will not map to a java parameter of type Object
          and a DECIMAL parameter will not map to an Object or Number.

          Show
          Daniel John Debrunner added a comment - Section 8.6 of part 13 is very precise on the method resolution. It even mentions "The Java Language Specification, Second Edition" in one section. Can you be more explicit on what you see in the "last paragraph of section 4.2.2" that leads you to believe that Java method resolution is followed? Note that Java method resolution is not followed by the SQL Standard, e.g. a VARCHAR() column will not map to a java parameter of type Object and a DECIMAL parameter will not map to an Object or Number.
          Hide
          Rick Hillegas added a comment -

          Thanks for helping me puzzle through these issues.

          Just to be clear, I don't want to tackle varargs in CREATE FUNCTION statements. I think that lies outside the SQL standard. However, I think that the Derby SQL interpreter should be able to resolve method calls which the Java compiler can handle. So, for example, given the following method declaration in a Java class

          public static String varArgsFunc( String... args )

          I think that the following bindings should be meaningful and should be resolved when the functions are actually invoked:

          create function twoArgs( arg1 varchar( 10 ), arg2 varchar( 10 ) )
          returns varchar( 40 )
          language java
          parameter style java
          no sql
          external name 'z.varArgsFunc'
          ;

          create function threeArgs( arg1 varchar( 10 ), arg2 varchar( 10 ), arg3 varchar( 10 ) )
          returns varchar( 40 )
          language java
          parameter style java
          no sql
          external name 'z.varArgsFunc'
          ;

          Part 13 of the SQL standard is a little vague, but the resolution rules in the last paragraph of section 4.2.2 lead me to believe that the intention is that the SQL interpreter should follow the method resolution rules used by Java itself. I may not know where to look, but it seems to me that the Standard is vague about which version of Java is intended.

          I have no problem classifying this as an improvement rather than a bug. However, I think that for customers who are using Java 5 or Java 6, the description of CREATE FUNCTION in the Reference Guide is vague and misleading enough that something ought to be done: either tighten up the documentation or implement varargs resolution. I think that the latter makes Derby more useful.

          Show
          Rick Hillegas added a comment - Thanks for helping me puzzle through these issues. Just to be clear, I don't want to tackle varargs in CREATE FUNCTION statements. I think that lies outside the SQL standard. However, I think that the Derby SQL interpreter should be able to resolve method calls which the Java compiler can handle. So, for example, given the following method declaration in a Java class public static String varArgsFunc( String... args ) I think that the following bindings should be meaningful and should be resolved when the functions are actually invoked: create function twoArgs( arg1 varchar( 10 ), arg2 varchar( 10 ) ) returns varchar( 40 ) language java parameter style java no sql external name 'z.varArgsFunc' ; create function threeArgs( arg1 varchar( 10 ), arg2 varchar( 10 ), arg3 varchar( 10 ) ) returns varchar( 40 ) language java parameter style java no sql external name 'z.varArgsFunc' ; Part 13 of the SQL standard is a little vague, but the resolution rules in the last paragraph of section 4.2.2 lead me to believe that the intention is that the SQL interpreter should follow the method resolution rules used by Java itself. I may not know where to look, but it seems to me that the Standard is vague about which version of Java is intended. I have no problem classifying this as an improvement rather than a bug. However, I think that for customers who are using Java 5 or Java 6, the description of CREATE FUNCTION in the Reference Guide is vague and misleading enough that something ought to be done: either tighten up the documentation or implement varargs resolution. I think that the latter makes Derby more useful.
          Hide
          Daniel John Debrunner added a comment -

          I don't see this as a bug, it's an improvement.

          Show
          Daniel John Debrunner added a comment - I don't see this as a bug, it's an improvement.
          Hide
          Daniel John Debrunner added a comment -

          One reason not to implement this is that method resolution is defined by the SQL Standard part 13, which does not cover 1.5 varargs since they were added later.

          Show
          Daniel John Debrunner added a comment - One reason not to implement this is that method resolution is defined by the SQL Standard part 13, which does not cover 1.5 varargs since they were added later.
          Hide
          Rick Hillegas added a comment -

          To show the current behavior, compile the attached z.java source file and then run the attached z.sql scripts.

          Show
          Rick Hillegas added a comment - To show the current behavior, compile the attached z.java source file and then run the attached z.sql scripts.

            People

            • Assignee:
              Rick Hillegas
              Reporter:
              Rick Hillegas
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development