Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-773

LineRecordReader can report non-zero progress while it is processing a compressed stream

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: task
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modifies LineRecordReader to report an approximate progress, instead of just returning 0, when using compressed streams.

      Description

      Currently, the LineRecordReader returns 0.0 from getProgress() for most inputs (since the "end" of the filesplit is set to Long.MAX_VALUE for compressed inputs). This can be improved to return a non-zero progress even for compressed streams (though it may not be very reflective of the actual progress).

      1. 773.patch
        4 kB
        Devaraj Das
      2. 773.patch
        4 kB
        Devaraj Das
      3. 773.3.patch
        10 kB
        Devaraj Das
      4. 773.2.patch
        6 kB
        Devaraj Das

        Activity

        Tom White made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Jothi Padmanabhan made changes -
        Release Note Modifies LineRecordReader to report an approximate progress, instead of just returning 0, when using compressed streams.
        Devaraj Das made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Devaraj Das made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj Das made changes -
        Attachment 773.3.patch [ 12416766 ]
        Devaraj Das made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Devaraj Das made changes -
        Attachment 773.2.patch [ 12415500 ]
        Devaraj Das made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj Das made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Devaraj Das made changes -
        Attachment 773.patch [ 12414093 ]
        Devaraj Das made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj Das made changes -
        Field Original Value New Value
        Attachment 773.patch [ 12413999 ]
        Devaraj Das created issue -

          People

          • Assignee:
            Devaraj Das
            Reporter:
            Devaraj Das
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development