Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-2547

Fix multi-process concurrence bug in UpdateCubeInfoXXStep

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: v2.0.0
    • Fix Version/s: Future
    • Component/s: Metadata
    • Labels:
      None

      Description

      There is a minor bug in "Update Cube Info" step when build a cube in distributed env, which will make this job failed, but the next auto merge job won't be affected and will succeed. So the cube data still is consistent and correct.

         Caused by: java.lang.IllegalStateException: Segments overlap: waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000] and waimai_dolphin_topic_flow_activity_expose_food_d_cube[20170405000000_20170412000000]
              at org.apache.kylin.cube.CubeValidator.validate(CubeValidator.java:85)
              at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:359)
              at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:386)
              at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:302)
              at org.apache.kylin.cube.CubeManager.mergeSegments(CubeManager.java:533)
              at org.apache.kylin.rest.service.CubeService.mergeCubeSegment(CubeService.java:635)
              at org.apache.kylin.rest.service.CubeService.updateOnNewSegmentReady(CubeService.java:587)
              at org.apache.kylin.rest.service.CubeServiceFastClassBySpringCGLIB17a07c0e.invoke()
              at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
              at org.springframework.aop.framework.CglibAopProxyDynamicAdvisedInterceptor.intercept(CglibAopProxy.java:629)
              at org.apache.kylin.rest.service.CubeServiceEnhancerBySpringCGLIB$$c6fabb3f.updateOnNewSegmentReady()
              at org.apache.kylin.rest.service.CacheService.rebuildCubeCache(CacheService.java:237)
              at org.apache.kylin.rest.service.CacheService.access$000(CacheService.java:62)
              at org.apache.kylin.rest.service.CacheService$1.afterCubeUpdate(CacheService.java:86)
              at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:305)
              at org.apache.kylin.cube.CubeManager.promoteNewlyBuiltSegments(CubeManager.java:735)
              at org.apache.kylin.engine.mr.steps.UpdateCubeInfoAfterBuildStep.doWork(UpdateCubeInfoAfterBuildStep.java:62)
              at org.apache.kylin.job.execution.AbstractExecutable.execute(AbstractExecutable.java:113)
              ... 6 more
      

        Attachments

          Activity

            People

            • Assignee:
              kangkaisen Kaisen Kang
              Reporter:
              kangkaisen Kaisen Kang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: