Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.21.1, 0.22.0
    • Fix Version/s: 0.21.1, 0.22.0, 0.23.0
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When I was testing HDFS-1114, the patch passed all tests except TestFiHFlush. Then, I tried to print out some debug messages, however, TestFiHFlush succeeded after added the messages.

      TestFiHFlush probably depends on the speed of BlocksMap. If BlocksMap is slow enough, then it will pass.

      1. HDFS-1206.patch
        0.7 kB
        Konstantin Boudnik
      2. HDFS-1206.patch
        0.7 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-22-branch #35 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-22-branch/35/ )
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/)

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #643 (See https://builds.apache.org/hudson/job/Hadoop-Hdfs-trunk/643/ )
          Hide
          Konstantin Boudnik added a comment -

          I have just commtted this to 0.21 branch and up

          Show
          Konstantin Boudnik added a comment - I have just commtted this to 0.21 branch and up
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Hide
          Konstantin Boudnik added a comment -

          Fixing a typo in an assert message

          Show
          Konstantin Boudnik added a comment - Fixing a typo in an assert message
          Hide
          Konstantin Boudnik added a comment -

          The attached patch solves the issue. Here's the reason for the intermittent failure we've been seen on and off:

          • test expects to find a pipeline in place when hflush is called.
          • as we are hflush'ing something like 1/10th of a block size after each write op. pipeline might get disassembled after 10th hflush is happening (in cases of files <= block_size)
          • the cases above are exactly where we see intermittent failures (0[1-2]_a)

          Thus, but fixing the assumption (i.e. we have to validate smthg on an existing pipeline) the test gets effectively fixed.

          Review anyone?

          Show
          Konstantin Boudnik added a comment - The attached patch solves the issue. Here's the reason for the intermittent failure we've been seen on and off: test expects to find a pipeline in place when hflush is called. as we are hflush'ing something like 1/10th of a block size after each write op. pipeline might get disassembled after 10th hflush is happening (in cases of files <= block_size) the cases above are exactly where we see intermittent failures (0 [1-2] _a) Thus, but fixing the assumption (i.e. we have to validate smthg on an existing pipeline) the test gets effectively fixed. Review anyone?
          Hide
          Konstantin Boudnik added a comment -

          My understanding is that "Fix version" needs to be updated after the fix, bit before.

          Show
          Konstantin Boudnik added a comment - My understanding is that "Fix version" needs to be updated after the fix, bit before.
          Hide
          Stephen Watt added a comment -

          @Nicholas - We need to add Fix Version information to this ticket so it will show up in the right place. JIRA isn't allowing me to do it. Can you update the ticket.

          Show
          Stephen Watt added a comment - @Nicholas - We need to add Fix Version information to this ticket so it will show up in the right place. JIRA isn't allowing me to do it. Can you update the ticket.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Talked to Cos. TestFiHFlush has some known problem.

          Show
          Tsz Wo Nicholas Sze added a comment - Talked to Cos. TestFiHFlush has some known problem.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Saw it failing again.

          Testcase: hFlushFi01_a took 4.553 sec
          	FAILED
          
          junit.framework.AssertionFailedError: 
          	at org.apache.hadoop.hdfs.TestFiHFlush.runDiskErrorTest(TestFiHFlush.java:56)
          	at org.apache.hadoop.hdfs.TestFiHFlush.hFlushFi01_a(TestFiHFlush.java:72)
          
          Show
          Tsz Wo Nicholas Sze added a comment - Saw it failing again. Testcase: hFlushFi01_a took 4.553 sec FAILED junit.framework.AssertionFailedError: at org.apache.hadoop.hdfs.TestFiHFlush.runDiskErrorTest(TestFiHFlush.java:56) at org.apache.hadoop.hdfs.TestFiHFlush.hFlushFi01_a(TestFiHFlush.java:72)

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development