Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-599

BufferedPositionedInputStream isn't buffered

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.2.0
    • Fix Version/s: 0.2.0
    • Component/s: impl
    • Labels:
      None

      Description

      org.apache.pig.impl.io.BufferedPositionedInputStream is not actually buffered. This is because it sits atop a FSDataInputStream (somewhere down the stack), which is buffered. So to avoid double buffering, which can be bad, BufferedPositionedInputStream was written without buffering. But the FSDataInputStream is far enough down the stack that it is still quite costly to call read() individually for each byte. A run through a profiler shows that a fair amount of time is being spent in BufferedPositionedInputStream.read().

        Attachments

        1. loadperf-2.patch
          6 kB
          Alan Gates
        2. loadperf.patch
          6 kB
          Alan Gates

          Activity

            People

            • Assignee:
              gates Alan Gates
              Reporter:
              gates Alan Gates
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: