Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3828

ArrayIndexOutOfBoundsException thrown when build a streaming cube with empty data in its first dimension

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v2.6.0
    • Fix Version/s: v2.6.1
    • Component/s: NRT Streaming
    • Labels:
      None

      Description

      The root cause is
      in org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper#doMap

          public void doMap(LongWritable key, BytesWritable value, Context context) throws IOException, InterruptedException {
              ByteBuffer buffer = ByteBuffer.wrap(value.getBytes(), 0, value.getLength());
              StreamingMessageRow row = streamingParser.parse(buffer).get(0);
              if (row == null) {
                  throw new IllegalArgumentException("");
              }
      
              data = StringUtil.join(row.getData(), delimiter);
              // output this row to value
              outValue.set(Bytes.toBytes(data));
              context.write(outKey, outValue);
          }
      

      Method StringUtil.join is used to join a collection of string to a single string with delimiter. But org.apache.kylin.common.util.StringUtil#join will ignore the leading empty strings in the colletion, source code is:

          public static String join(Iterable<String> parts, String separator) {
              StringBuilder buf = new StringBuilder();
              for (String p : parts) {
                  if (buf.length() > 0)
                      buf.append(separator);
                  buf.append(p);
              }
              return buf.toString();
          }
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                nichunen nichunen
                Reporter:
                nichunen nichunen
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: