Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1466

FileInputFormat should save #input-files in JobConf

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.22.0
    • 0.21.0
    • client
    • None
    • Reviewed
    • Added a private configuration variable mapreduce.input.num.files, to store number of input files being processed by M/R job.

    Description

      We already track the amount of data consumed by MR applications (MAP_INPUT_BYTES), alongwith, it would be useful to #input-files from the client-side for analysis. Along the lines of MAPREDUCE-1403, it would be easy to stick in the JobConf during job-submission.

      Attachments

        1. MAPREDUCE-1466_yhadoop20.patch
          9 kB
          Arun Murthy
        2. MAPREDUCE-1466_yhadoop20-1.patch
          9 kB
          Hemanth Yamijala
        3. MAPREDUCE-1466_yhadoop20-2.patch
          9 kB
          Hemanth Yamijala
        4. MAPREDUCE-1466_yhadoop20-3.patch
          9 kB
          Hemanth Yamijala
        5. mr-1466-trunk-v1.patch
          10 kB
          Luke Lu
        6. mr-1466-trunk-v2.patch
          13 kB
          Luke Lu
        7. mr-1466-trunk-v3.patch
          17 kB
          Luke Lu
        8. mr-1466-trunk-v4.patch
          17 kB
          Luke Lu
        9. mr-1466-trunk-v5.patch
          17 kB
          Luke Lu

        Activity

          People

            vicaya Luke Lu
            acmurthy Arun Murthy
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: