Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-1111

Data loss is observed when atlas is restarted while hive_table metadata ingestion into kafka topic is in-progress

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.8-incubating
    • Component/s: None
    • Labels:
      None

      Description

      During atlas stop, the graph is shutdown first and then the services like NotificationHookConsumer is shutdown. After graph is shutdown and before NotificationHookConsumer is shutdown, the message handling fails as the graph is down, but the NotificationHookConsumer commits the message(updating the offset). So, the messages during this time are lost

        Attachments

        1. ATLAS-1111.5.patch
          55 kB
          Suma Shivaprasad
        2. ATLAS-1111.4.patch
          51 kB
          Shwetha GS
        3. ATLAS-1111.3.patch
          51 kB
          Shwetha GS
        4. ATLAS-1111.2.patch
          24 kB
          Shwetha GS
        5. ATLAS-1111.1.patch
          17 kB
          Suma Shivaprasad
        6. ATLAS-1111.patch
          12 kB
          Shwetha GS

          Activity

            People

            • Assignee:
              shwethags Shwetha GS
              Reporter:
              sharmadhas Sharmadha S

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment