Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1104

Long dimension value cause ArrayIndexOutOfBoundsException

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: v1.3.0, v1.5.0
    • Component/s: None
    • Labels:
      None

      Description

      I get ArrayIndexOutOfBoundsException while building cube, and the error is caused by one dimension(I think measure can not be so long) value is longer than 4096 bytes.
      I check source code and find the reason:
      1、it specify max split length is 4096 when create BytesSplitter object in setup.
      2、in BytesSplitter.split function which split all items in one row with delimiter, and using System.arraycopy(bytes, offset, split.value, 0, length); copy every item to a SplittedBytes object named split. and before this line , it execute split.length = length; to set the split length.
      3、while executing the copy code, it throw ArrayIndexOutOfBoundsException because the real item length is bigger than 4096!
      4、in mapper function, the exception will catched and execute handleErrorRecord function to handle it, However, in this function it will call use splits again(in toString function), ArrayIndexOutOfBoundsException will throw again.
      5、There is not one catch this exception, which casue the mapper execute failed.

      That is all my situation and analysis, I will modify source code to continue my job building and I will upload my patch later.

        Attachments

        1. 0001-.patch
          3 kB
          fengYu
        2. 0001-bug.patch
          3 kB
          fengYu
        3. 0001-bytes-split-max-length-limit.patch
          2 kB
          fengYu

          Activity

            People

            • Assignee:
              feng_xiao_yu fengYu
              Reporter:
              feng_xiao_yu fengYu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: