Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1344

Incorrect inferred precision when BigDecimal value is less than 1

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.8.0
    • 1.9.0
    • None
    • None

    Description

      Method makeExactLiteral(BigDecimal) in RexBuilder infers the incorrect precision when decimal < 1 e.g. for 0.06 it infers the type to be Decimal(1,2) instead of Decimal(3,2).

        /**
         * Creates a numeric literal.
         */
        public RexLiteral makeExactLiteral(BigDecimal bd) {
          RelDataType relType;
          int scale = bd.scale();
          long l = bd.unscaledValue().longValue();
          assert scale >= 0;
          assert scale <= typeFactory.getTypeSystem().getMaxNumericScale() : scale;
          assert BigDecimal.valueOf(l, scale).equals(bd);
          if (scale == 0) {
            if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE)) {
              relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
            } else {
              relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
            }
          } else {
            int precision = bd.unscaledValue().abs().toString().length();
            relType =
                typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
          }
          return makeExactLiteral(bd, relType);
        }
      

      Attachments

        Issue Links

          Activity

            People

              jcamachorodriguez Jesus Camacho Rodriguez
              jcamachorodriguez Jesus Camacho Rodriguez
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: