Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-4164

[Atlas: Spooling] Tables created after spooling are created before the spooled tables when there is multiple frequent restart in kafka brokers

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.1.0
    • 3.0.0
    • atlas-core

    Description

      Scenario:

      • Stop kafka broker
      • Create a few (20) tables save the prefix (abc_table_1, abc_table_2, ... abc_table_n)
      • Make sure the data is spooled
      • Start kafka and create a few more tables (xyz_table_1, xyz_table_2, ... xyz_table_n)
      • Wait for 5 mins for the tables to reflect in atlas

      In this case we expect all the abc_table_* to be created before xyz_table_1, meaning all the spooled tables are created before the tables that are created after spooling.

       

      Observation:

      createTime of some spooled tables is greater than the create time of the xyz_table_1

       

      Sample data:

      createTime for tables that are spooled:

      [1613573518284, 1613573531470, 1613573531861, 1613573529446, 1613573543253, 1613573525390, 1613573525950, 1613573517796, 1613573518284, 1613573522629, 1613573513524, 1613573524856, 1613573518992, 1613573519477, 1613573519947, 1613573521737, 1613573514066, 1613573514555, 1613573515065, 1613573515605]

      createTime for tables that are created after spooling:

      [1613573540582, 1613573541300, 1613573551691, 1613573552628, 1613573553356, 1613573555478, 1613573556275, 1613573556940, 1613573557763, 1613573558659, 1613573560673, 1613573561363, 1613573562310, 1613573563096, 1613573564004, 1613573566533, 1613573567602, 1613573568439, 1613573569379, 1613573570202] 

      We expect all spooled tables to have createTime smaller than the tables created after spooling.

      But 1613573543253 (Spooled tabled create time) is greater than 1613573540582 (table created after spooling)
      which means, the table created after spooling is created before spooled table

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            radhikak Radhika Kundam
            dharshmk Dharshana M Krishnamoorthy
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment