Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: Heterogeneous Storage (HDFS-2832)
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      There are a few more test issues in TestBlockReport caused by the earlier changes.

      testBlockReport_07 fails and it looks like a test issue.

      Running org.apache.hadoop.hdfs.server.datanode.TestBlockReport
      Tests run: 9, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 110.824 sec <<< FAILURE! - in org.apache.hadoop.hdfs.server.datanode.TestBlockReport
      blockReport_07(org.apache.hadoop.hdfs.server.datanode.TestBlockReport)  Time elapsed: 19.314 sec  <<< FAILURE!
      java.lang.AssertionError: Wrong number of Corrupted blocks expected:<1> but was:<0>
              at org.junit.Assert.fail(Assert.java:93)
              at org.junit.Assert.failNotEquals(Assert.java:647)
              at org.junit.Assert.assertEquals(Assert.java:128)
              at org.junit.Assert.assertEquals(Assert.java:472)
              at org.apache.hadoop.hdfs.server.datanode.TestBlockReport.blockReport_07(TestBlockReport.java:461)
      

      TestBPOfferService fails due to missing implementation of SimulatedFSDataset#getStorageReports.

      2013-10-28 16:24:33,775 ERROR datanode.DataNode (BPServiceActor.java:run(719)) - Exception in BPOfferService for Block pool fake bpid (Datanode Uuid null) service to 0.0.0.0/0.0.0.0:0
      java.lang.UnsupportedOperationException  at org.apache.hadoop.hdfs.server.datanode.SimulatedFSDataset.getStorageReports(SimulatedFSDataset.java:1005)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.sendHeartBeat(BPServiceActor.java:478)  at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:566)
        at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:717)
        at java.lang.Thread.run(Thread.java:695)
      
      1. h5437.08.patch
        20 kB
        Arpit Agarwal
      2. h5437.07.patch
        19 kB
        Arpit Agarwal
      3. h5437.05.patch
        18 kB
        Arpit Agarwal
      4. h5437.04.patch
        17 kB
        Arpit Agarwal
      5. h5437.03.patch
        17 kB
        Arpit Agarwal

        Activity

        Hide
        Arpit Agarwal added a comment -

        Some refactoring of the test case. Also added two @VisibleForTesting methods to BlockListAsLongs.

        TestBlockReport#blockReport_07 will fail on a different assertion now due to HDFS-5439.

        Show
        Arpit Agarwal added a comment - Some refactoring of the test case. Also added two @VisibleForTesting methods to BlockListAsLongs . TestBlockReport#blockReport_07 will fail on a different assertion now due to HDFS-5439 .
        Hide
        Arpit Agarwal added a comment -

        Including a trivial fix in SimulatedFSDataset#getStorageReports.

        Show
        Arpit Agarwal added a comment - Including a trivial fix in SimulatedFSDataset#getStorageReports .
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The BlockListAsLongs methods are only lightly used in the tests. I suggested

        • change setBlockLength to something like corruptBlockLengthForTesting(int i), which will get block length, generate a random length and set block length.
        • change setGenerationStamp in a similar way.
        • remove getBlockId, getBlockLen and getBlockGenStamp since they are not needed anymore.
        Show
        Tsz Wo Nicholas Sze added a comment - The BlockListAsLongs methods are only lightly used in the tests. I suggested change setBlockLength to something like corruptBlockLengthForTesting(int i), which will get block length, generate a random length and set block length. change setGenerationStamp in a similar way. remove getBlockId, getBlockLen and getBlockGenStamp since they are not needed anymore.
        Hide
        Arpit Agarwal added a comment -

        Thanks for taking a look Nicholas!

        Updated the patch with your feedback. I also needed a trivial fix in LocatedBlock.java and BPServiceActor.java since I used the wrong LocatedBlock constructor when merging with HDFS-4949 yesterday.

        Show
        Arpit Agarwal added a comment - Thanks for taking a look Nicholas! Updated the patch with your feedback. I also needed a trivial fix in LocatedBlock.java and BPServiceActor.java since I used the wrong LocatedBlock constructor when merging with HDFS-4949 yesterday.
        Hide
        Arpit Agarwal added a comment -

        Missed removing an unused constructor, new patch attached.

        Show
        Arpit Agarwal added a comment - Missed removing an unused constructor, new patch attached.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good. Thanks, Arpit.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Thanks, Arpit.
        Hide
        Arpit Agarwal added a comment -

        Thanks Nicholas. I committed this to branch HDFS-2832.

        Show
        Arpit Agarwal added a comment - Thanks Nicholas. I committed this to branch HDFS-2832 .

          People

          • Assignee:
            Arpit Agarwal
            Reporter:
            Arpit Agarwal
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development