[CALCITE-5877] AssertionError during MOD operation if result scale is greater than maximum numeric scale

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.35.0
• Fix Version/s: 1.36.0
• Component/s: core
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Mihai Budiu
• Created: 27/Jul/23 23:35
• Updated: 10/Nov/23 11:34
• Resolved: 02/Aug/23 09:02

Description

Adding the following test to RelToSqlConverterTest produces an AssertionFailure:

@Test void testNumericScaleMod() {
    final String sql = "SELECT MOD(CAST(2 AS DECIMAL(39, 20)), 2)";
    final String expected = "";
    sql(sql).withPostgresqlModifiedDecimalTypeSystem()
        .ok(expected);
  }

The exception is here:

java.lang.AssertionError
	at org.apache.calcite.rel.type.RelDataTypeSystem.deriveDecimalModType(RelDataTypeSystem.java:383)
	at org.apache.calcite.sql.type.ReturnTypes.lambda$static$11(ReturnTypes.java:849)
	at org.apache.calcite.sql.type.SqlTypeTransformCascade.inferReturnType(SqlTypeTransformCascade.java:58)
	at org.apache.calcite.sql.type.SqlReturnTypeInferenceChain.inferReturnType(SqlReturnTypeInferenceChain.java:55)

In this line:

int scale = Math.max(s1, s2);
assert scale <= getMaxNumericScale();

I suspect that doing a Math.min with the getMaxNumericScale (instead of an Assert) is sufficient to fix this particular problem. The question is whether the compiler should really enforce the invariant that this assertion checks.

Issue Links

• links to: GitHub Pull Request #3341