Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4757

Impossible precision for decimal datatype in kylin if the source column is numeric in postgres

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Skill Level:
      Newcomer (Easy) - Everyone can do this

      Description

      Using Postgres as data source, columns of datatype numeric without a explicitly set precision and scale become "decimal(256)" in kylin. 256 comes from the value of "kylin.source.hive.default-varchar-precision" (despite using jdbc as datasource). This causes an error when building a cube.

      The problem can be fixed in "org.apache.kylin.source.jdbc.extensible.JdbcExplorer" class (https://github.com/apache/kylin/tree/ master/source-jdbc/src/main/java/org/apache/kylin/source/jdbc/extensible/JdbcExplorer.java#L111 )

      The csize variable defined in line 96 receives the value 131089 from columns.getInt("COLUMN_SIZE");

      I fixed it by modifying the following part in the source file starting after line 111:

      int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize : -1;                
      precision = Math.min(precision, KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision());                
      int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
      

      to this:

       

      int precision = (SqlUtil.isPrecisionApplicable(kylinType) && csize > 0) ? csize : -1;
      
      int maxPrecision = precision;
      if (kylinType.equals("char")) {
          maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultCharPrecision();
      } else if (kylinType.equals("varchar")) {
          maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultVarcharPrecision();
      } else if ((kylinType.equals("decimal") || kylinType.equals("numeric"))) {
          maxPrecision = KylinConfig.getInstanceFromEnv().getDefaultDecimalPrecision();
      }
      precision = Math.min(precision, maxPrecision);
      
      int scale = (SqlUtil.isScaleApplicable(kylinType) && digits > 0) ? digits : -1;
      

       

       

        Attachments

          Activity

            People

            • Assignee:
              xxyu Xiaoxiang Yu
              Reporter:
              andreoid André Luís da Rosa de Lima

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment