Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2616

[Gridmix] InputStriper should smartly switch between compressed and uncompressed files based on the simulated job's input data characteristics

    XMLWordPrintableJSON

    Details

    • Tags:
      gridmix compression-emulation

      Description

      Currently, all the Gridmix input data files are located at <gridmix-io-dir>/input (<gridmix-io-dir> is expected as a CLI parameter). When compression emulation is enabled, Gridmix will check for compressed files (based on suffixes) in the input folder. Gridmix will bail out if there are no compressed input files. If the input folder consists of a mix of compressed and uncompressed input files, then Gridmix will only use compressed input files for all the jobs. Gridmix should smartly assign
      1. uncompressed input files for jobs the don't need input decompression
      2. compressed input files for jobs that need input decompression

        Attachments

          Activity

            People

            • Assignee:
              amar_kamat Amar Kamat
              Reporter:
              amar_kamat Amar Kamat
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated: