Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3603

Setting spill threshold to 100% fails to detect spill for records

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If io.sort.record.percent is set to 1.0, the simultaneous collection and spill is disabled. However, if one exhausts the offset allocation before the serialization allocation, the limit will not be detected and the write will block forever.

        Attachments

        1. 3603-0.patch
          11 kB
          Christopher Douglas
        2. 3603-1.patch
          18 kB
          Christopher Douglas
        3. 3603-2.patch
          18 kB
          Christopher Douglas

          Activity

            People

            • Assignee:
              cdouglas Christopher Douglas
              Reporter:
              cdouglas Christopher Douglas
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: