[TS-941] invalid cast of off_t math to int

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.1
• Component/s: Cache

Environment

Stock debian amd64 2.6.34.7 kernel modified to increase XEN domU max RAM to 96Gb

Stock build with:
./configure --enable-layout=Debian --sysconfdir=/etc/trafficserver --libdir=
/usr/lib/trafficserver --with-user=root --with-group=root --enable-debug --enabl
e-static-libts CFLAGS= CXXFLAGS= --enable-wccp

• Assignee: Leif Hedstrom
• Reporter: Bart
• Created: 01/Sep/11 21:44
• Updated: 16/Sep/11 18:01
• Resolved: 02/Sep/11 22:11

Description

iocore/CacheWrite.cc:Vol::evac_range calculates its looping values from 64bit off_t values but stores them in 32bit integers:

CacheWrite.cc#1086711:694-700

 
int
Vol::evac_range(off_t low, off_t high, int evac_phase)
{
  int s = offset_to_vol_offset(this, low);
  int e = offset_to_vol_offset(this, high);
  int si = dir_offset_evac_bucket(s);
  int ei = dir_offset_evac_bucket(e);

When Vol::start and/or the low high parameters get large enough these loop values become truncated and potentially negative causing a general protection fault as it attempts to access memory addresses below the valid range.