Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5918

LineRecordReader can return the same decompressor to CodecPool multiple times

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.3.0
    • 2.7.0
    • None
    • None

    Description

      LineRecordReader can return the same decompressor to CodecPool multiple times if method close() called multiple times. In this case CodecPool doesn't guarantee that it always return different decompressors. This issue can cause some difficult reproducible and difficult diagnosable bugs in Hadoop based programs.

      Attachments

        1. MAPREDUCE-5918.1.patch
          3 kB
          Sergey Murylev
        2. MAPREDUCE-5918.2.patch
          7 kB
          Sergey Murylev
        3. MAPREDUCE-5918.3.patch
          7 kB
          Sergey Murylev
        4. MAPREDUCE-5918.4.patch
          14 kB
          Sergey Murylev

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-11424 Guard against MAPREDUCE-5918 by ensuring RecordReader is only closed once in *HadoopRDD

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Web Link Review Board

          Activity

            People

              phoenix13 Sergey Murylev
              phoenix13 Sergey Murylev
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: