Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6683

Restarting the Catalog without restarting Impalad and SS can block topic updates

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: Impala 2.12.0
    • Fix Version/s: Impala 3.0, Impala 2.12.0
    • Component/s: Catalog
    • Labels:
      None
    • Epic Color:
      ghx-label-7

      Description

      After restarting the Catalog without restarting the other services topic updates are getting stuck

      From SS

      I0315 17:20:44.305951 120990 statestore.cc:242] Preparing initial catalog-update topic update for impalad@va1031.foo:22000. Size = 50.18 MB
      I0315 17:20:44.305981 120974 statestore.cc:242] Preparing initial catalog-update topic update for impalad@vd1120.foo:22000. Size = 50.18 MB
      I0315 17:20:44.305982 120975 statestore.cc:242] Preparing initial catalog-update topic update for impalad@vc1518.foo:22000. Size = 50.18 MB
      I0315 17:20:44.343500 121001 statestore.cc:652] Received request for different delta base of topic: catalog-update from: impalad@vd1107.foo:22000 subscriber from_version: 0
      I0315 17:20:44.355831 120963 statestore.cc:242] Preparing initial catalog-update topic update for impalad@ve1134.foo:22000. Size = 50.18 MB
      I0315 17:20:44.355840 120983 statestore.cc:242] Preparing initial catalog-update topic update for impalad@vd1329.foo:22000. Size = 50.18 MB
      

      From Impalad

      E0315 17:20:36.468525  7355 impala-server.cc:1381] There was an error processing the impalad catalog update. Requesting a full topic update to recover: CatalogException: Detected catalog service ID change. Aborting updateCatalog()
      E0315 17:20:40.778920  7355 impala-server.cc:1381] There was an error processing the impalad catalog update. Requesting a full topic update to recover: CatalogException: Detected catalog service ID change. Aborting updateCatalog()
      I0315 17:20:42.951370 11846 StmtMetadataLoader.java:193] Waiting for table metadata. Waited for 170 catalog updates and 340149ms. Tables remaining: [metadata_benchmarks.80_partitions_250k_files]
      I0315 17:21:04.954602 11846 StmtMetadataLoader.java:214] Re-sending prioritized load request. Waited for 180 catalog updates and 362152ms.
      I0315 17:21:04.955175 11846 FeSupport.java:274] Requesting prioritized load of table(s): metadata_benchmarks.80_partitions_250k_files
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tianyiwang Tianyi Wang
                Reporter:
                mmokhtar Mostafa Mokhtar
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: