Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1387

Streaming cubing doesn't generate cuboids files on HDFS, cause cube merge failure

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.4.0
    • Component/s: NRT Streaming
    • Labels:
      None

      Description

      In an early version of Kylin streaming, the HBaseMROutput2.java will use HBase Scan API to do the merge. Later Yang refactored that into HBaseMROutput2Transition.java, while keep using the cuboids files on HDFS to do the merge. But the StreamingBatchBuilder didn't generate the cuboids, which causes the merge failed to run.

      Solution: as a workaround, the StreamingBatchBuilder should leave cuboids files when building micro segments.

        Attachments

          Activity

            People

            • Assignee:
              shaofengshi Shao Feng Shi
              Reporter:
              shaofengshi Shao Feng Shi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: