Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2616

Can't create Unicode literal by RelBuilder

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.17.0
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      Test in RelBuilderTest to reproduce the issue:

      @Test public void testScanWithFilterByUnicodeValue() {
        final RelBuilder builder = RelBuilder.create(config().build());
        RelNode root =
            builder.scan("EMP")
                .filter(
                    builder.call(SqlStdOperatorTable.EQUALS,
                        builder.field("ENAME"),
                        builder.literal("Петро ピーター")
                    )
                )
                .build();
      }
      

       Result:

      org.apache.calcite.runtime.CalciteException: Failed to encode 'Петро ピーター' in character set 'ISO-8859-1'

      Possible workaround: create saffron.properties with the following property

      saffron.default.charset=UTF-16LE

      But UTF-8 will not work as a value of this property, see SqlUtil.translateCharacterSetName

      Related code:

      • SqlUtil.translateCharacterSetName(charsetName)
      • RelDataTypeFactoryImpl.getDefaultCharset()
      • SaffronProperties

       Could it be considered to switch defaults from "ISO-8859-1" to "UTF-8" in SaffronProperties?

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                anha Anton Haidai
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: