Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-4136

Real time streaming segment building may save the same cube segment twice in some case

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: v3.0.0
    • Fix Version/s: Future
    • Component/s: None
    • Labels:
      None

      Description

      Real time streaming segment building may save the same cube segment whit new state twice in some case , that 's lead to can't build the segment success.

      // exception
      2019-08-13 20:22:09,644 ERROR [http-bio-8081-exec-64] coordinator.Coordinator:1127 : streaming job submit fail, cubeName:rt_common_task_state_change_v3_cl2 segment:20190809195000_20190809200000
      java.lang.IllegalStateException: Segments overlap: rt_common_task_state_change_v3_cl2[20190809195000_20190809200000] and rt_common_task_state_change_v3_cl2[20190809195000_20190809200000]
      at org.apache.kylin.metadata.model.Segments.validate(Segments.java:426)
      at org.apache.kylin.cube.CubeManager.updateCubeWithRetry(CubeManager.java:388)
      at org.apache.kylin.cube.CubeManager.updateCube(CubeManager.java:301)
      at org.apache.kylin.cube.CubeManager$SegmentAssist.appendSegment(CubeManager.java:742)
      at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:662)
      at org.apache.kylin.cube.CubeManager.appendSegment(CubeManager.java:650)
      at org.apache.kylin.stream.coordinator.Coordinator.triggerSegmentBuild(Coordinator.java:1110)
      at org.apache.kylin.stream.coordinator.Coordinator.tryFindAndBuildSegment(Coordinator.java:982)
      at org.apache.kylin.stream.coordinator.Coordinator.segmentRemoteStoreComplete(Coordinator.java:294)
      at org.apache.kylin.rest.service.StreamingCoordinatorService.onSegmentRemoteStoreComplete(StreamingCoordinatorService.java:114)
      at org.apache.kylin.rest.controller.StreamingCoordinatorController.segmentRemoteStoreComplete(StreamingCoordinatorController.java:184)
      at sun.reflect.GeneratedMethodAccessor106.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:498)
      at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
      at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
      at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
      at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
      at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
      at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
      at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
      at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
      at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
      at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:650)
      at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
      at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
      

      //debug

        Attachments

        1. image-2019-08-13-21-03-53-045.png
          688 kB
          wangxiaojing

          Activity

            People

            • Assignee:
              wangxiaojing wangxiaojing
              Reporter:
              wangxiaojing wangxiaojing
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: