Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3828

ArrayIndexOutOfBoundsException thrown when build a streaming cube with empty data in its first dimension

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v2.6.0
    • Fix Version/s: v2.6.1
    • Component/s: NRT Streaming
    • Labels:
      None

      Description

      The root cause is
      in org.apache.kylin.source.kafka.hadoop.KafkaFlatTableMapper#doMap

          public void doMap(LongWritable key, BytesWritable value, Context context) throws IOException, InterruptedException {
              ByteBuffer buffer = ByteBuffer.wrap(value.getBytes(), 0, value.getLength());
              StreamingMessageRow row = streamingParser.parse(buffer).get(0);
              if (row == null) {
                  throw new IllegalArgumentException("");
              }
      
              data = StringUtil.join(row.getData(), delimiter);
              // output this row to value
              outValue.set(Bytes.toBytes(data));
              context.write(outKey, outValue);
          }
      

      Method StringUtil.join is used to join a collection of string to a single string with delimiter. But org.apache.kylin.common.util.StringUtil#join will ignore the leading empty strings in the colletion, source code is:

          public static String join(Iterable<String> parts, String separator) {
              StringBuilder buf = new StringBuilder();
              for (String p : parts) {
                  if (buf.length() > 0)
                      buf.append(separator);
                  buf.append(p);
              }
              return buf.toString();
          }
      

        Attachments

          Activity

            People

            • Assignee:
              nichunen nichunen
              Reporter:
              nichunen nichunen

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment