Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9361

Strictly define the expected behavior of filesystem APIs and write tests to verify compliance

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.4.0, 3.0.0-alpha1
    • Fix Version/s: 2.5.0
    • Component/s: fs, test
    • Labels:
      None
    • Target Version/s:

      Description

      FileSystem and FileContract aren't tested rigorously enough -while HDFS gets tested downstream, other filesystems, such as blobstore bindings, don't.

      The only tests that are common are those of FileSystemContractTestBase, which HADOOP-9258 shows is incomplete.

      I propose

      1. writing more tests which clarify expected behavior
      2. testing operations in the interface being in their own JUnit4 test classes, instead of one big test suite.
      3. Having each FS declare via a properties file what behaviors they offer, such as atomic-rename, atomic-delete, umask, immediate-consistency -test methods can downgrade to skipped test cases if a feature is missing.

        Attachments

        1. HADOOP-9361-018.patch
          383 kB
          Steve Loughran
        2. HADOOP-9361-017.patch
          383 kB
          Steve Loughran
        3. HADOOP-9361-016.patch
          380 kB
          Steve Loughran
        4. HADOOP-9361-015.patch
          380 kB
          Steve Loughran
        5. HADOOP-9361-014.patch
          378 kB
          Steve Loughran
        6. HADOOP-9361-013.patch
          377 kB
          Steve Loughran
        7. HADOOP-9361-012.patch
          304 kB
          Steve Loughran
        8. HADOOP-9361-011.patch
          311 kB
          Steve Loughran
        9. HADOOP-9361-009.patch
          327 kB
          Steve Loughran
        10. HADOOP-9361-008.patch
          323 kB
          Steve Loughran
        11. HADOOP-9361-007.patch
          316 kB
          Steve Loughran
        12. HADOOP-9361-006.patch
          295 kB
          Steve Loughran
        13. HADOOP-9361-005.patch
          276 kB
          Steve Loughran
        14. HADOOP-9361-004.patch
          248 kB
          Steve Loughran
        15. HADOOP-9361-003.patch
          225 kB
          Steve Loughran
        16. HADOOP-9361-002.patch
          146 kB
          Steve Loughran
        17. HADOOP-9361-001.patch
          65 kB
          Steve Loughran
        18. HADOOP-9361.awang-addendum.patch
          96 kB
          Andrew Wang

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                35 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 48h
                  48h
                  Remaining:
                  Remaining Estimate - 48h
                  48h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified