Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-773

LineRecordReader can report non-zero progress while it is processing a compressed stream

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: task
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modifies LineRecordReader to report an approximate progress, instead of just returning 0, when using compressed streams.

      Description

      Currently, the LineRecordReader returns 0.0 from getProgress() for most inputs (since the "end" of the filesplit is set to Long.MAX_VALUE for compressed inputs). This can be improved to return a non-zero progress even for compressed streams (though it may not be very reflective of the actual progress).

        Attachments

        1. 773.patch
          4 kB
          Devaraj Das
        2. 773.patch
          4 kB
          Devaraj Das
        3. 773.2.patch
          6 kB
          Devaraj Das
        4. 773.3.patch
          10 kB
          Devaraj Das

          Activity

            People

            • Assignee:
              devaraj Devaraj Das
              Reporter:
              devaraj Devaraj Das
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: