Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.22.0
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Most HDFS tests are functional tests that test a feature end to end by running a mini cluster. We should add more tests like TestReplication that attempt to stress individual classes in isolation, ie by stubbing out dependencies without running a mini cluster. This allows for more fine-grain testing and making tests run much more quickly because they avoid the cost of cluster setup and teardown. If it makes sense to use another framework besides junit we should standardize with MAPREDUCE-1050.

      1. HDFS669.patch
        5 kB
        Eli Collins
      2. HDFS-669.patch
        4 kB
        Konstantin Boudnik
      3. HDFS-669.patch
        8 kB
        Konstantin Boudnik
      4. HDFS-669.patch
        9 kB
        Konstantin Boudnik
      5. HDFS-669.patch
        8 kB
        Konstantin Boudnik
      6. HDFS-669.patch
        8 kB
        Konstantin Boudnik
      7. HDFS-669.patch
        8 kB
        Konstantin Boudnik
      8. HDFS-669.patch
        8 kB
        Konstantin Boudnik
      9. HDFS-669.sh
        0.1 kB
        Konstantin Boudnik

        Issue Links

          Activity

          Eli Collins created issue -
          Eli Collins made changes -
          Field Original Value New Value
          Summary Use mock objects for unit tests Add unit tests
          Description Like MAPREDUCE-1050 but for HDFS. If using framework makes sense we should standardize across Hadoop projects. Using mock objects will allow for fine grain unit tests (ie write tests for particular class without creating a MiniDFSCluster by creating mock objects for some/all of the classes dependencies). I'll work on an example patch for preliminary review. Most HDFS tests are functional tests that test a feature end to end by running a mini cluster. We should add more tests like TestReplication that attempt to stress individual classes in isolation, ie by stubbing out dependencies without running a mini cluster. This allows for more fine-grain testing and making tests run much more quickly because they avoid the cost of cluster setup and teardown. If it makes sense to use another framework besides junit we should standardize with MAPREDUCE-1050.
          Eli Collins made changes -
          Attachment HDFS669.patch [ 12421597 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue is related to MAPREDUCE-1050 [ MAPREDUCE-1050 ]
          Konstantin Boudnik made changes -
          Assignee Eli Collins [ eli ] Konstantin Boudnik [ cos ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424541 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424545 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424620 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424634 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424660 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424661 ]
          Konstantin Boudnik made changes -
          Link This issue blocks HDFS-519 [ HDFS-519 ]
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Affects Version/s 0.21.0 [ 12314046 ]
          Affects Version/s 0.22.0 [ 12314241 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.patch [ 12424733 ]
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Konstantin Boudnik made changes -
          Attachment HDFS-669.sh [ 12424759 ]
          Jakob Homan made changes -
          Hadoop Flags [Reviewed]
          Konstantin Boudnik made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Konstantin Boudnik made changes -
          Summary Add unit tests Add unit tests framework (Mockito)
          Konstantin Boudnik made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.22.0 [ 12314241 ]
          Resolution Fixed [ 1 ]
          Konstantin Boudnik made changes -
          Fix Version/s 0.21.0 [ 12314046 ]
          Component/s build [ 12312925 ]
          Component/s test [ 12312916 ]
          Konstantin Boudnik made changes -
          Link This issue is cloned as HDFS-820 [ HDFS-820 ]
          Tom White made changes -
          Fix Version/s 0.22.0 [ 12314241 ]
          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Konstantin Boudnik
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development