Hadoop Common
  1. Hadoop Common
  2. HADOOP-1381

The distance between sync blocks in SequenceFiles should be configurable rather than hard coded to 2000 bytes

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: io
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Made sync interval of sequencefiles configurable and raised default from 100 bytes to 100 kilobytes, to optimize for large files.

      Description

      Currently SequenceFiles put in sync blocks every 2000 bytes. It would be much better if it was configurable with a much higher default (1mb or so?).

      1. HADOOP-1381.r5.diff
        9 kB
        Harsh J
      2. HADOOP-1381.r5.diff
        9 kB
        Harsh J
      3. HADOOP-1381.r4.diff
        9 kB
        Harsh J
      4. HADOOP-1381.r3.diff
        9 kB
        Harsh J
      5. HADOOP-1381.r2.diff
        8 kB
        Harsh J
      6. HADOOP-1381.r1.diff
        6 kB
        Harsh J

        Activity

          People

          • Assignee:
            Harsh J
            Reporter:
            Owen O'Malley
          • Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:

              Development