[TS-3878] icp.config is not reloadable

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.2.3, 5.3.1
• Fix Version/s: 6.1.0
• Component/s: ICP
• Backport to Version: 5.3.3
• Assignee: Leif Hedstrom
• Reporter: Gota Adachi
• Created: 01/Sep/15 02:16
• Updated: 04/Feb/16 18:49
• Resolved: 08/Sep/15 17:30

Description

Currently, traffic_server(or traffic_manager) is able to recognize any changes of icp.config which exec traffic_line -x.
But reloading function doesn't work.

# cat /usr/local/var/log/trafficserver/traffic.out 
[E. Mgmt] log ==> [TrafficManager] using root directory '/usr/local'
traffic_server: using root directory '/usr/local'
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) On=2, MultiCast=0, Timeout=1 LocalCacheLookup=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) StaleLookup=0, ReplyToUnknowPeer=0, DefaultReplyPort=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [0]: Type=L IP=192.168.56.98:3130 PPort=0 Host=localhost
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [0]: MC ON=0 MC_IP=*Not IP address [0]* MC_TTL=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [1]: Type=S IP=192.168.56.22:3130 PPort=8080 Host=
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [1]: MC ON=0 MC_IP=0.0.0.0 MC_TTL=1
# vi /usr/local/etc/trafficserver/icp.config (change something)
# /usr/local/bin/traffic_ctl config reload
# cat /usr/local/var/log/trafficserver/traffic.out
[E. Mgmt] log ==> [TrafficManager] using root directory '/usr/local'
traffic_server: using root directory '/usr/local'
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) On=2, MultiCast=0, Timeout=1 LocalCacheLookup=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) StaleLookup=0, ReplyToUnknowPeer=0, DefaultReplyPort=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [0]: Type=L IP=192.168.56.98:3130 PPort=0 Host=localhost
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [0]: MC ON=0 MC_IP=*Not IP address [0]* MC_TTL=0
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [1]: Type=S IP=192.168.56.22:3130 PPort=8080 Host=
[Sep  1 08:19:59.681] Server {0x2af442a36040} DEBUG: (icp) [1]: MC ON=0 MC_IP=0.0.0.0 MC_TTL=1

It is caused that termination of the icp process function doesn't work.

icp processor open socket for waiting query, and when finish the icp processor,
it sends a dummy queries towards the socket opened at ICPProcessor::CancelPendingReads().

void
ICPProcessor::CancelPendingReads()
{
  // Cancel pending ICP read by sending a bogus message to
  //  the local ICP port.
 
  // ...snip... 
  Peer *lp = GetLocalPeer();
  r->_sendMsgHdr.msg_name = (caddr_t) & (lp->GetSendChan())->addr;
  r->_sendMsgHdr.msg_namelen = sizeof((lp->GetSendChan())->addr);
  udpNet.sendmsg_re(r, r, lp->GetSendFD(), &r->_sendMsgHdr);
}

But this function's implemetation is broken because (lp->GetSendChan())->addr is not contain
local endpoint address, and it contains all 0. Therefore udpNet.sendmsg_re() always fail.

This problem is able to fix if only that set local endpoint address appropriately.

The patch that fix this issue is below(for currently master branch):

diff --git a/proxy/ICP.cc b/proxy/ICP.cc
index 4f3c768..7eacebb 100644
--- a/proxy/ICP.cc
+++ b/proxy/ICP.cc
@@ -2209,8 +2209,10 @@ ICPProcessor::CancelPendingReads()
   r->_ICPmsg.h.version = ~r->_ICPmsg.h.version; // bogus message
 
   Peer *lp = GetLocalPeer();
-  r->_sendMsgHdr.msg_name = (caddr_t) & (lp->GetSendChan())->addr;
-  r->_sendMsgHdr.msg_namelen = sizeof((lp->GetSendChan())->addr);
+  IpEndpoint local_endpoint;
+  ats_ip_copy(&local_endpoint.sa, lp->GetIP());
+  r->_sendMsgHdr.msg_name = (caddr_t) & local_endpoint;
+  r->_sendMsgHdr.msg_namelen = sizeof(local_endpoint);
   udpNet.sendmsg_re(r, r, lp->GetSendFD(), &r->_sendMsgHdr);
 }
 

Testing configurations:

records.config

#...
CONFIG proxy.config.icp.enabled INT 2
CONFIG proxy.config.icp.icp_interface STRING eth1
CONFIG proxy.config.icp.icp_port INT 3130
CONFIG proxy.config.icp.multicast_enabled INT 0
CONFIG proxy.config.icp.query_timeout INT 1
#...
#output debugging messages for icp (minimal)
CONFIG proxy.config.diags.debug.enabled INT 1
CONFIG proxy.config.diags.debug.tags STRING icp

icp.config

:192.168.56.22:2:8080:3130:0:0.0.0.0:1:
#:192.168.56.23:2:8080:3130:0:0.0.0.0:1:

it works like this:

# /etc/init.d/trafficserver start
Starting Apache Traffic Server:                            [  OK  ]
# cat /usr/local/var/log/trafficserver/traffic.out
traffic_server: using root directory '/usr/local'
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) On=2, MultiCast=0, Timeout=1 LocalCacheLookup=0
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) StaleLookup=0, ReplyToUnknowPeer=0, DefaultReplyPort=0
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) [0]: Type=L IP=192.168.56.98:3130 PPort=0 Host=localhost
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) [0]: MC ON=0 MC_IP=*Not IP address [0]* MC_TTL=0
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) [1]: Type=S IP=192.168.56.23:3130 PPort=8080 Host=
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) [1]: MC ON=0 MC_IP=0.0.0.0 MC_TTL=1

# vi /usr/local/etc/trafficserver/icp.config (any changes that such as swap commented out.)
# /usr/local/bin/traffic_ctl config reload
# (wait around 10 seconds...)
# cat /usr/local/var/log/trafficserver/traffic.out
...
[Sep  1 08:28:32.404] Server {0x2ab665d0b040} DEBUG: (icp) [1]: MC ON=0 MC_IP=0.0.0.0 MC_TTL=1
[Sep  1 08:33:42.441] Server {0x2ab6666a5700} DEBUG: (icp) Received msg from invalid sender [192.168.56.98:3130]
[Sep  1 08:33:42.454] Server {0x2ab6666a5700} DEBUG: (icp) On=2, MultiCast=0, Timeout=1 LocalCacheLookup=0
[Sep  1 08:33:42.454] Server {0x2ab6666a5700} DEBUG: (icp) StaleLookup=0, ReplyToUnknowPeer=0, DefaultReplyPort=0
[Sep  1 08:33:42.454] Server {0x2ab6666a5700} DEBUG: (icp) [0]: Type=L IP=192.168.56.98:3130 PPort=0 Host=localhost
[Sep  1 08:33:42.454] Server {0x2ab6666a5700} DEBUG: (icp) [0]: MC ON=0 MC_IP=*Not IP address [0]* MC_TTL=0
[Sep  1 08:33:42.454] Server {0x2ab6666a5700} DEBUG: (icp) [1]: Type=S IP=192.168.56.22:3130 PPort=8080 Host=
[Sep  1 08:33:42.455] Server {0x2ab6666a5700} DEBUG: (icp) [1]: MC ON=0 MC_IP=0.0.0.0 MC_TTL=1

That should be able to watch changes has been applied.