Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1344

Incorrect inferred precision when BigDecimal value is less than 1

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.8.0
    • Fix Version/s: 1.9.0
    • Component/s: None
    • Labels:
      None

      Description

      Method makeExactLiteral(BigDecimal) in RexBuilder infers the incorrect precision when decimal < 1 e.g. for 0.06 it infers the type to be Decimal(1,2) instead of Decimal(3,2).

        /**
         * Creates a numeric literal.
         */
        public RexLiteral makeExactLiteral(BigDecimal bd) {
          RelDataType relType;
          int scale = bd.scale();
          long l = bd.unscaledValue().longValue();
          assert scale >= 0;
          assert scale <= typeFactory.getTypeSystem().getMaxNumericScale() : scale;
          assert BigDecimal.valueOf(l, scale).equals(bd);
          if (scale == 0) {
            if ((l >= Integer.MIN_VALUE) && (l <= Integer.MAX_VALUE)) {
              relType = typeFactory.createSqlType(SqlTypeName.INTEGER);
            } else {
              relType = typeFactory.createSqlType(SqlTypeName.BIGINT);
            }
          } else {
            int precision = bd.unscaledValue().abs().toString().length();
            relType =
                typeFactory.createSqlType(SqlTypeName.DECIMAL, precision, scale);
          }
          return makeExactLiteral(bd, relType);
        }
      

        Issue Links

          Activity

          Hide
          jcamachorodriguez Jesus Camacho Rodriguez added a comment -

          Julian Hyde, could you take a look at the PR?

          https://github.com/apache/calcite/pull/266

          Thanks

          Show
          jcamachorodriguez Jesus Camacho Rodriguez added a comment - Julian Hyde , could you take a look at the PR? https://github.com/apache/calcite/pull/266 Thanks
          Hide
          julianhyde Julian Hyde added a comment -

          +1

          Show
          julianhyde Julian Hyde added a comment - +1
          Show
          jcamachorodriguez Jesus Camacho Rodriguez added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/39baed5 . Thanks for reviewing Julian Hyde
          Hide
          jcamachorodriguez Jesus Camacho Rodriguez added a comment -

          Resolved in release 1.9.0 (2016-09-22)

          Show
          jcamachorodriguez Jesus Camacho Rodriguez added a comment - Resolved in release 1.9.0 (2016-09-22)

            People

            • Assignee:
              jcamachorodriguez Jesus Camacho Rodriguez
              Reporter:
              jcamachorodriguez Jesus Camacho Rodriguez
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development