Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-16153

Allow TeraGen to use schema-specific output committer

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.1.1
    • None
    • None
    • None

    Description

      TeraGen is hard-coded to use FileOutputCommitter to commit the job. This patch is to allow TeraGen to use schema-specific committer for optimization. For example, using S3A committers for s3a:// object storage.

      Attachments

        1. HADOOP-16153.patch
          1 kB
          Yifeng Jiang

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue HADOOP-16058 S3A tests to include Terasort

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue MAPREDUCE-7091 Terasort on S3A to switch to new committers

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              uprush Yifeng Jiang
              uprush Yifeng Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: