Pig
  1. Pig
  2. PIG-3505

Make AvroStorage sync interval take default from io.file.buffer.size

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11
    • Fix Version/s: 0.13.0
    • Component/s: None
    • Labels:
      None

      Description

      The default sync interval is 16K which is very bad for bzip compression which can take bigger chunk of data for compression. Hadoop's Bzip2code uses io.file.buffer.size as the buffer size. Most tuned environments have it set to 128K which gives better compression.

      1. PIG-3505-1.patch
        4 kB
        Rohini Palaniswamy

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        2m 12s 1 Rohini Palaniswamy 08/Oct/13 00:15
        Patch Available Patch Available Resolved Resolved
        35d 20h 13m 1 Rohini Palaniswamy 12/Nov/13 20:29
        Resolved Resolved Closed Closed
        236d 21h 38m 1 Daniel Dai 07/Jul/14 18:07
        Daniel Dai made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Rohini Palaniswamy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Rohini Palaniswamy added a comment -

        Committed to trunk. Thanks for the review Cheolsoo.

        Show
        Rohini Palaniswamy added a comment - Committed to trunk. Thanks for the review Cheolsoo.
        Hide
        Cheolsoo Park added a comment -

        +1.

        Show
        Cheolsoo Park added a comment - +1.
        Rohini Palaniswamy made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Rohini Palaniswamy made changes -
        Field Original Value New Value
        Attachment PIG-3505-1.patch [ 12607269 ]
        Rohini Palaniswamy created issue -

          People

          • Assignee:
            Rohini Palaniswamy
            Reporter:
            Rohini Palaniswamy
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development