Traffic Server
  1. Traffic Server
  2. TS-46

increase cache partition size to avoid write-aggregation seeks

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0a
    • Component/s: Cache
    • Labels:
      None
    • Environment:

      all

      Description

      Currently cache partitions are 8GB. Because each has a write pointer, this means on a decent size
      disk there are a hundred or more write pointers which means a lot of seeking for writes
      (which really should be aggregated). The proposal is to make it possible to have large partitions
      (up to .5 PB) by increasing the side of the cache directory entry from 8 to 10 bytes and using the
      additional bits for the offset.

      1. ts-partition-size-jp-v1.patch
        51 kB
        John Plevyak
      2. ts-partition-size-jp-v2.patch
        74 kB
        John Plevyak
      3. ts-partition-size-jp-v3.patch
        83 kB
        John Plevyak
      4. ts-partition-size-jp-v4.patch
        141 kB
        John Plevyak

        Activity

        Hide
        John Plevyak added a comment -

        This is an initial patch. It currently does unaligned accesses to memory, but it has been tested for 10+ hours on x86 32-bit and a smoke test on x86_64. For your reviewing/testing pleasure.

        Show
        John Plevyak added a comment - This is an initial patch. It currently does unaligned accesses to memory, but it has been tested for 10+ hours on x86 32-bit and a smoke test on x86_64. For your reviewing/testing pleasure.
        Hide
        George Paul added a comment -

        I have tested this patch for 3+hrs on ubuntu 8.04LTS x86_64 w/ 2 disks and I have not noticed any issues in forward proxy mode. Preliminary indications indicate performance has improved >50% with these changes with roughly same CPU utilization with logging off but more thorough perf-testing is needed.
        -George

        Show
        George Paul added a comment - I have tested this patch for 3+hrs on ubuntu 8.04LTS x86_64 w/ 2 disks and I have not noticed any issues in forward proxy mode. Preliminary indications indicate performance has improved >50% with these changes with roughly same CPU utilization with logging off but more thorough perf-testing is needed. -George
        Hide
        John Plevyak added a comment -

        Version 2. This patch doesn't do unaligned reads and add support for large fragments (16MB).
        The aggregation buffer is only 2MB which effectively limits objects to just less than that.
        This patch increased the size of the aggregation buffer (to 2MB) as well as the evacuation
        buffers and also added a new fast allocate aligned buffer size of 64K.

        Show
        John Plevyak added a comment - Version 2. This patch doesn't do unaligned reads and add support for large fragments (16MB). The aggregation buffer is only 2MB which effectively limits objects to just less than that. This patch increased the size of the aggregation buffer (to 2MB) as well as the evacuation buffers and also added a new fast allocate aligned buffer size of 64K.
        Hide
        John Plevyak added a comment -

        Updates the previous patches to include a fix for sync order in recovery and
        to introduce the O_DSYNC flag if available. This flag was previously not
        necessary because older professional SCSI disks did not use a write-back cache.

        Show
        John Plevyak added a comment - Updates the previous patches to include a fix for sync order in recovery and to introduce the O_DSYNC flag if available. This flag was previously not necessary because older professional SCSI disks did not use a write-back cache.
        Hide
        John Plevyak added a comment -

        This patch includes support for do_io_pread for efficient support of Range requests.
        It also includes support for large objects, although full support will require changing
        the VIO and VConnection interfaces to use ink64 sizes.

        Show
        John Plevyak added a comment - This patch includes support for do_io_pread for efficient support of Range requests. It also includes support for large objects, although full support will require changing the VIO and VConnection interfaces to use ink64 sizes.
        Hide
        John Plevyak added a comment -

        Checked in to branches/dev to be checked into the 2.1 fork when it happens.

        Show
        John Plevyak added a comment - Checked in to branches/dev to be checked into the 2.1 fork when it happens.

          People

          • Assignee:
            John Plevyak
            Reporter:
            John Plevyak
          • Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 168h
              168h
              Remaining:
              Remaining Estimate - 168h
              168h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development