Hadoop Common
  1. Hadoop Common
  2. HADOOP-9361

Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.4.0
    • Fix Version/s: 2.5.0
    • Component/s: fs, test
    • Labels:
      None

      Description

      FileSystem and FileContract aren't tested rigorously enough -while HDFS gets tested downstream, other filesystems, such as blobstore bindings, don't.

      The only tests that are common are those of FileSystemContractTestBase, which HADOOP-9258 shows is incomplete.

      I propose

      1. writing more tests which clarify expected behavior
      2. testing operations in the interface being in their own JUnit4 test classes, instead of one big test suite.
      3. Having each FS declare via a properties file what behaviors they offer, such as atomic-rename, atomic-delete, umask, immediate-consistency -test methods can downgrade to skipped test cases if a feature is missing.
      1. HADOOP-9361-001.patch
        65 kB
        Steve Loughran
      2. HADOOP-9361-002.patch
        146 kB
        Steve Loughran
      3. HADOOP-9361-003.patch
        225 kB
        Steve Loughran
      4. HADOOP-9361-004.patch
        248 kB
        Steve Loughran
      5. HADOOP-9361-005.patch
        276 kB
        Steve Loughran
      6. HADOOP-9361-006.patch
        295 kB
        Steve Loughran
      7. HADOOP-9361-007.patch
        316 kB
        Steve Loughran
      8. HADOOP-9361-008.patch
        323 kB
        Steve Loughran
      9. HADOOP-9361-009.patch
        327 kB
        Steve Loughran
      10. HADOOP-9361-011.patch
        311 kB
        Steve Loughran
      11. HADOOP-9361-012.patch
        304 kB
        Steve Loughran
      12. HADOOP-9361-013.patch
        377 kB
        Steve Loughran
      13. HADOOP-9361-014.patch
        378 kB
        Steve Loughran
      14. HADOOP-9361.awang-addendum.patch
        96 kB
        Andrew Wang
      15. HADOOP-9361-015.patch
        380 kB
        Steve Loughran
      16. HADOOP-9361-016.patch
        380 kB
        Steve Loughran
      17. HADOOP-9361-017.patch
        383 kB
        Steve Loughran
      18. HADOOP-9361-018.patch
        383 kB
        Steve Loughran

        Issue Links

          Activity

            People

            • Assignee:
              Steve Loughran
              Reporter:
              Steve Loughran
            • Votes:
              0 Vote for this issue
              Watchers:
              35 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 48h
                48h
                Remaining:
                Remaining Estimate - 48h
                48h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development