Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2943

TsTabletManagerITest.TestTableStats flaky due to WAL/cmeta term disagreement

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 1.11.0
    • None
    • consensus, test
    • None

    Description

      This new test failed in a strange (and worrying) way:

      /home/jenkins-slave/workspace/kudu-master/1/src/kudu/integration-tests/ts_tablet_manager-itest.cc:753: Failure
      Failed
      Bad status: Corruption: Unable to start RaftConsensus: The last op in the WAL with id 3.4 has a term (3) that is greater than the latest recorded term, which is 2
      

      From a brief dig through the code, looks like this means the current term as per the on-disk cmeta file is older than the term in the latest WAL op.

      I can believe that this is somehow due to InternalMiniCluster exercising clean shutdown paths that aren't well tested or robust, but it'd be nice to determine that with certainty.

      I've attached the full test log.

      Attachments

        1. ts_tablet_manager-itest.txt
          721 kB
          Adar Dembo

        Issue Links

          Activity

            People

              Unassigned Unassigned
              adar Adar Dembo
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: