Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10494

Ensure Jets3tFileSystemStore respects URI path for block files

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 3.0.1
    • Component/s: fs/s3
    • Labels:

      Description

      The Jets3tFileSystemStore will currently store all block files in the root of an S3 bucket. If you specify a path in your OutputFormat, those paths are created, and some of the supporting meta files are stored there, but all block files will reside in the root of the bucket.

      We tracked this down to this method here, which does not use the provided URI from the the job: https://github.com/apache/hadoop-common/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/s3/Jets3tFileSystemStore.java#L381

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              MarkJGreene Mark J Greene
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: