Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-773

LineRecordReader can report non-zero progress while it is processing a compressed stream

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.21.0
    • task
    • None
    • Reviewed
    • Modifies LineRecordReader to report an approximate progress, instead of just returning 0, when using compressed streams.

    Description

      Currently, the LineRecordReader returns 0.0 from getProgress() for most inputs (since the "end" of the filesplit is set to Long.MAX_VALUE for compressed inputs). This can be improved to return a non-zero progress even for compressed streams (though it may not be very reflective of the actual progress).

      Attachments

        1. 773.2.patch
          6 kB
          Devaraj Das
        2. 773.3.patch
          10 kB
          Devaraj Das
        3. 773.patch
          4 kB
          Devaraj Das
        4. 773.patch
          4 kB
          Devaraj Das

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ddas Devaraj Das
            ddas Devaraj Das
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment