Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2656

ErrorType/TabletServerDiskErrorTest.TestRandomOpSequence is flaky

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Cannot Reproduce
    • Affects Version/s: 1.8.0
    • Fix Version/s: NA
    • Component/s: test
    • Labels:
      None

      Description

      Here's the stack trace:

      cfile_reader.cc:593] Check failed: io_context 
          @     0x7f3c7ec31c37 gsignal at ??:0
          @     0x7f3c7ec35028 abort at ??:0
          @     0x7f3c890b664e kudu::cfile::CFileReader::HandleCorruption() at ??:0
          @     0x7f3c890b4546 kudu::cfile::CFileReader::InitOnce() at ??:0
          @     0x7f3c890cbfbb kudu::cfile::CFileReader::Init()::$_0::operator()() at ??:0
          @     0x7f3c890cbe7c _ZZN4kudu14KuduOnceLambda4InitIZNS_5cfile11CFileReader4InitEPKNS_2fs9IOContextEE3$_0EENS_6StatusET_ENKUlvE_clEv at ??:0
          @     0x7f3c898e2a80 __GI___pthread_once at ??:0
          @     0x7f3c890cba4a _ZSt9call_onceIZN4kudu14KuduOnceLambda4InitIZNS0_5cfile11CFileReader4InitEPKNS0_2fs9IOContextEE3$_0EENS0_6StatusET_EUlvE_JEEvRSt9once_flagOSB_DpOT0_ at ??:0
          @     0x7f3c890b7a42 kudu::KuduOnceLambda::Init<>() at ??:0
          @     0x7f3c890b3ad8 kudu::cfile::CFileReader::Init() at ??:0
          @     0x7f3c8dbdec05 kudu::tablet::DeltaFileReader::InitOnce() at ??:0
          @     0x7f3c8dbe3bd8 kudu::tablet::DeltaFileReader::Init()::$_1::operator()() at ??:0
          @     0x7f3c8dbe3a3c _ZZN4kudu14KuduOnceLambda4InitIZNS_6tablet15DeltaFileReader4InitEPKNS_2fs9IOContextEE3$_1EENS_6StatusET_ENKUlvE_clEv at ??:0
          @     0x7f3c898e2a80 __GI___pthread_once at ??:0
          @     0x7f3c8dbe360a _ZSt9call_onceIZN4kudu14KuduOnceLambda4InitIZNS0_6tablet15DeltaFileReader4InitEPKNS0_2fs9IOContextEE3$_1EENS0_6StatusET_EUlvE_JEEvRSt9once_flagOSB_DpOT0_ at ??:0
          @     0x7f3c8dbde9e2 kudu::KuduOnceLambda::Init<>() at ??:0
          @     0x7f3c8dbde88d kudu::tablet::DeltaFileReader::Init() at ??:0
          @     0x7f3c8dc7e447 kudu::tablet::DeltaTracker::ValidateDeltaOrder() at ??:0
          @     0x7f3c8dc7fe46 kudu::tablet::DeltaTracker::AtomicUpdateStores() at ??:0
          @     0x7f3c8dc2a6e0 kudu::tablet::MajorDeltaCompaction::UpdateDeltaTracker() at ??:0
          @     0x7f3c8daf54d1 kudu::tablet::DiskRowSet::MajorCompactDeltaStoresWithColumnIds() at ??:0
          @     0x7f3c8daf4905 kudu::tablet::DiskRowSet::MajorCompactDeltaStores() at ??:0
          @     0x7f3c8d966409 kudu::tablet::Tablet::CompactWorstDeltas() at ??:0
          @     0x7f3c8da178eb kudu::tablet::MajorDeltaCompactionOp::Perform() at ??:0
          @     0x7f3c8372eef7 kudu::MaintenanceManager::LaunchOp() at ??:0
          @     0x7f3c8373fde3 boost::_bi::bind_t<>::operator()() at ??:0
      

      From code inspection, it appears that ValidateDeltaOrder (only used in DEBUG builds) may Init() new delta files, and since it passes a null IOContext, we eventually trigger a CHECK in the CFileReader code.

        Attachments

          Activity

            People

            • Assignee:
              awong Andrew Wong
              Reporter:
              adar Adar Dembo
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: