Apache Drill
  1. Apache Drill
  2. DRILL-949

support of reading ctrl-A delimited text file in storage-plugins.json

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: 0.4.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently Drill doesn't support ctrl-A ('\001') type of delimiter in storage-plugins.json file:

      I have this in the format:

      "ctrlA" :

      { type: "text", extensions: [ "txt" ], delimiter: "\\001" }

      ,

      and getting error while Drillbit starts:

      Exception in thread "51a92118-3fab-4b4e-bdf9-2a7057ee52e4:frag:0:0" java.lang.IllegalArgumentException: Only single character delimiter supported

        Activity

        Hide
        Sudheesh Katkam added a comment -

        The escape sequence for ctrl-A is \001. This format is not supported by json (http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf ). Please use \u0001 for ctrl-A; this is valid java and json.

        Show
        Sudheesh Katkam added a comment - The escape sequence for ctrl-A is \001. This format is not supported by json ( http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf ). Please use \u0001 for ctrl-A; this is valid java and json.
        Hide
        Vivian Summers added a comment -

        "\001" won't work, drillbit failed to start:

        16:50:19.697 [main] ERROR o.a.d.e.store.StoragePluginRegistry - Failure while reading storage plugins data.
        com.fasterxml.jackson.core.JsonParseException: Unrecognized character escape '0' (code 48)
        at [Source: java.io.StringReader@7b3c3a4e; line: 25, column: 25]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1369) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:574) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._decodeEscaped(ReaderBasedJsonParser.java:1781) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:1528) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:1505) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getValueAsString(ReaderBasedJsonParser.java:221) ~[jackson-core-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:29) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:157) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:123) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:106) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:434) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:157) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:123) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:106) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:434) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:347) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:976) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:276) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888) ~[jackson-databind-2.2.0.jar:2.2.0]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2034) ~[jackson-databind-2.2.0.jar:2.2.0]
        at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:139) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:113) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:128) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:62) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:49) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:71) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]

        Show
        Vivian Summers added a comment - "\001" won't work, drillbit failed to start: 16:50:19.697 [main] ERROR o.a.d.e.store.StoragePluginRegistry - Failure while reading storage plugins data. com.fasterxml.jackson.core.JsonParseException: Unrecognized character escape '0' (code 48) at [Source: java.io.StringReader@7b3c3a4e; line: 25, column: 25] at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1369) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.base.ParserMinimalBase._handleUnrecognizedCharacterEscape(ParserMinimalBase.java:574) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._decodeEscaped(ReaderBasedJsonParser.java:1781) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString2(ReaderBasedJsonParser.java:1528) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._finishString(ReaderBasedJsonParser.java:1505) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getValueAsString(ReaderBasedJsonParser.java:221) ~ [jackson-core-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:29) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.StringDeserializer.deserialize(StringDeserializer.java:11) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:157) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:123) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:106) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:434) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.impl.FieldProperty.deserializeAndSet(FieldProperty.java:107) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:157) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:123) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:113) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:82) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:106) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringMap(MapDeserializer.java:434) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:314) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:26) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:449) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:347) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:976) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:276) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888) ~ [jackson-databind-2.2.0.jar:2.2.0] at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2034) ~ [jackson-databind-2.2.0.jar:2.2.0] at org.apache.drill.exec.store.StoragePluginRegistry.createPlugins(StoragePluginRegistry.java:139) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT] at org.apache.drill.exec.store.StoragePluginRegistry.init(StoragePluginRegistry.java:113) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.run(Drillbit.java:128) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:62) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.start(Drillbit.java:49) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT] at org.apache.drill.exec.server.Drillbit.main(Drillbit.java:71) [drill-java-exec-1.0.0-m2-incubating-SNAPSHOT-rebuffed.jar:1.0.0-m2-incubating-SNAPSHOT]
        Hide
        Sudheesh Katkam added a comment - - edited

        Please try "\001" instead of "\ \001" as the delimiter.

        Show
        Sudheesh Katkam added a comment - - edited Please try "\001" instead of "\ \001" as the delimiter.

          People

          • Assignee:
            Jacques Nadeau
            Reporter:
            Vivian Summers
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development