Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12177 [Umbrella] Update and extend filesystem specification
  3. HADOOP-11601

Enhance FS spec & tests to mandate FileStatus.getBlocksize() >0 for non-empty files

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: fs, test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HADOOP-11584 has shown that the contract tests are not validating that FileStatus.getBlocksize() must be >0 for any analytics jobs to partition workload correctly.

      Clarify in text and add test to do this. Test MUST be designed to work against eventually consistent filesystems where getFileStatus() may not be immediately visible, by retrying operation if FS declares it is an object store.

        Attachments

        1. HADOOP-11601-001.patch
          11 kB
          Steve Loughran
        2. HADOOP-11601-002.patch
          8 kB
          Steve Loughran
        3. HADOOP-11601-003.patch
          9 kB
          Steve Loughran
        4. HADOOP-11601-004.patch
          9 kB
          Steve Loughran
        5. HADOOP-11601-005.patch
          9 kB
          Steve Loughran
        6. HADOOP-11601-006.patch
          9 kB
          Mingliang Liu
        7. HADOOP-11601-007.patch
          9 kB
          Mingliang Liu

          Issue Links

            Activity

              People

              • Assignee:
                stevel@apache.org Steve Loughran
                Reporter:
                stevel@apache.org Steve Loughran
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1h
                  1h
                  Remaining:
                  Remaining Estimate - 1h
                  1h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified