Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1555

LogBlockManager FlushDataAsync() synchronously flushes metadata

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0
    • 0.10.0
    • fs, perf
    • None

    Description

      I'm looking at the time spent by a flush in a YCSB workload, and most of the time is spent in stacks like:

      #0 0x0000003a1acdfd9a in sync_file_range () from /lib64/libc.so.6
      #1 0x00000000018a83e4 in ?? ()
      #2 0x00000000018ee741 in kudu::pb_util::WritablePBContainerFile::Flush() ()
      #3 0x000000000187ae56 in kudu::fs::internal::LogWritableBlock::FlushDataAsync() ()
      #4 0x00000000017b177e in kudu::cfile::CFileWriter::FinishAndReleaseBlock(kudu::fs::ScopedWritableBlockCloser*) ()
      #5 0x0000000000905206 in kudu::tablet::MultiColumnWriter::FinishAndReleaseBlocks(kudu::fs::ScopedWritableBlockCloser*) ()

      The purpose of doing the FlushDataAsync() is so that the disks work in parallel, but currently it appears that this is ended up blocked in synchronous metadata flushes on each disk in turn. I think this should probably perform an async metadata flush, since we later relay on a SyncMetadata() call to ensure it is complete.

      Attachments

        Activity

          People

            tlipcon Todd Lipcon
            tlipcon Todd Lipcon
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: