Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1344

Incorrect inferred precision when BigDecimal value is less than 1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Method makeExactLiteral(BigDecimal) in RexBuilder infers the incorrect precision when decimal < 1 e.g. for 0.06 it infers the type to be Decimal(1,2) instead of Decimal(3,2).

        /**
         * Creates a numeric literal.
         */
        public RexLiteral makeExactLiteral(BigDecimal bd) {
          RelDataType relType;
          int scale = bd.scale();
          long l = bd.unscaledValue().longValue();
          assert scale >= 0;
          assert scale <= typeFactory.getTypeSystem().getMaxNumericScale() : scale;
          assert BigDecimal.valueOf(l, scale).equals(bd);
          if (scale == 0) {
            if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE)) {
              relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
            } else {
              relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
            }
          } else {
            int precision = bd.unscaledValue().abs().toString().length();
            relType =
                typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
          }
          return makeExactLiteral(bd, relType);
        }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jcamachorodriguez Jesus Camacho Rodriguez
                Reporter:
                jcamachorodriguez Jesus Camacho Rodriguez
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: