Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7475

Make TestLazyPersistFiles#testLazyPersistBlocksAreSaved deterministic

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestLazyPersistFiles#testLazyPersistBlocksAreSaved verifies that the memory block is saved in lazy persist directory after some sleep with the following code.

          // Sleep for a short time to allow the lazy writer thread to do its job
          Thread.sleep(6 * LAZY_WRITER_INTERVAL_SEC * 1000);
      

      However, the sleep only guarantees an async lazy persist task is scheduled. The task running on a per volume thread pool thread can take longer to finish persisting. It will be more deterministic to wait the pending lazy persist task count reduce to 0 before verifying the block file exist in lazy persist dir.

        Attachments

        1. HDFS-7475.0.patch
          5 kB
          Xiaoyu Yao
        2. HDFS-7475.1.patch
          5 kB
          Xiaoyu Yao
        3. HDFS-7475.2.patch
          5 kB
          Xiaoyu Yao

          Activity

            People

            • Assignee:
              xyao Xiaoyu Yao
              Reporter:
              xyao Xiaoyu Yao
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: