Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-1500

streaming_fillgap cause out of memory

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.4.0, v1.5.1
    • Component/s: Streaming
    • Labels:
      None

      Description

      streaming_fillgap.sh will try to fill gap when there is some error of the streaming job. But no matter how long the gap it is, how big the missing data it is, kylin tried to fill gap in one thread, which sometimes cause out of memory error. Need streaming_fillgap.sh to be more intelligent based on gap time range and data volume.

        Attachments

          Activity

            People

            • Assignee:
              yaho Zhong Yanghong
              Reporter:
              qianqiaoneng qianqiaoneng
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: