Bug 51247

Summary: Enhance mod_proxy and _balancer with worker status flag to only accept sticky session routes
Product: Apache httpd-2 Reporter: Keith Mashinter <kmashint>
Component: mod_proxyAssignee: Apache HTTPD Bugs Mailing List <bugs>
Status: RESOLVED FIXED    
Severity: enhancement CC: cameron.stokes
Priority: P2    
Version: 2.2.19   
Target Milestone: ---   
Hardware: All   
OS: All   
URL: https://docs.google.com/open?id=0B5zLg35jBZLTMFN3c2JBc014RVU
Attachments: Enhance mod_proxy and _balancer with worker status flag to only accept sticky session routes
mod_proxy_route_only_for_trunk.patch
mod_proxy_route_only_for_2_2_19.patch
mod_proxy_route_only_for_2_2_19
mod_proxy_route_only_for_2_2_21.patch
Backport of this idea adopted as a Drain status from 2.3/2.4 to Apache-2.2
2.2.21 Win32 Release build to add Drain status to proxy balancer workers
2.2.22 Win32 Release build to add Drain status to proxy balancer workers

Description Keith Mashinter 2011-05-23 18:42:44 UTC
Created attachment 27047 [details]
Enhance mod_proxy and _balancer with worker status flag to only accept sticky session routes

This enhancement, actually SVN Patched against 2.2.19, provides a worker status flag to set a proxy worker as only accepting requests with sticky session routes, e.g. only accept requests with a .route such as Cookie JSESSIONID=xxx.tc2.

This allows for a graceful fade-out of servers when their sessions are removed; they continue to receive requests for their sticky session routes but are passed over for requests with no specified route, just as if they were disabled.  In other words, route-only workers are only enabled for sticky session routes.

Intended use (Tomcat JSESSIONID noted here but could be PHPSESSIONID, Ruby _session_id, or anything with cookie or request-parameter based session ids):
1. An Apache rev-proxy running for multiple Tomcats.
2. To fade-out a Tomcat for maintenance, set route-only enabled in the balancer-manager or reload the configuration with the worker status +R.
(This depends on Tomcat web-apps delete session cookies, see further below.)
3. Check on the balancer-manager or its Tomcat worker even a few minutes / hours, and when it seems to have completed old sessions you can mark it fully disabled/
4. Once done maintenance, you can then set route-only disabled (status -R) and fully enable the worker again.

To delete a JSESSIONID Cookie from a Servlet, you need to specify the same Domain and Path as the original Cookie and setMaxAge=0 as in the typical example below but you should check on your own Domain and Path when a Cookie is created, e.g. watch the Cookie headers in Firefox Firebug.

	// To delete a Cookie setMaxAge(0) and also any original domain and path if specified.
	Cookie ck = new Cookie("JSESSIONID", null);
	//ck.setDomain("");
	ck.setPath(request.getContextPath());
	ck.setMaxAge(0);
	response.addCookie(ck);
Comment 1 Keith Mashinter 2011-05-25 12:29:20 UTC
Created attachment 27060 [details]
mod_proxy_route_only_for_trunk.patch

This is a patch in theory only, since I'm still fighting to get the trunk to compile, but wanted to share the intent and will replace once compiled.
Comment 2 Keith Mashinter 2011-05-25 12:53:18 UTC
Created attachment 27061 [details]
mod_proxy_route_only_for_2_2_19.patch

Re-attach patch for 2.2.19 marking it specifically as a patch.
Comment 3 Keith Mashinter 2011-05-27 01:13:54 UTC
Created attachment 27073 [details]
mod_proxy_route_only_for_2_2_19

Just making the constant forward compatible with the trunk for 2.3.x.
Comment 4 Keith Mashinter 2011-08-21 15:48:26 UTC
A similar idea is in https://issues.apache.org/bugzilla/show_bug.cgi?id=48841 :
"mod_proxy only allows a BalancerMember's loadfactor to be set to a value
between 1 and 100.  Ideally, loadfactor should be allowed to be set to 0 so
that a BalancerMember can be removed from new traffic but allowed to keep
servicing existing sessions (i.e. bleeding off sessions) or to be able to start
a BalancerMember and test it before letting new traffic access it."

This is also a useful way to gracefully drop a reverse-proxy back-end server from use.
Comment 5 Keith Mashinter 2011-08-21 15:54:26 UTC
A similar idea is in https://issues.apache.org/bugzilla/show_bug.cgi?id=48841 :
"mod_proxy only allows a BalancerMember's loadfactor to be set to a value
between 1 and 100.  Ideally, loadfactor should be allowed to be set to 0 so
that a BalancerMember can be removed from new traffic but allowed to keep
servicing existing sessions (i.e. bleeding off sessions) or to be able to start
a BalancerMember and test it before letting new traffic access it."

This is also a useful way to gracefully drop a reverse-proxy back-end server from use.
Comment 6 Keith Mashinter 2011-08-21 15:55:54 UTC
A similar idea is in https://issues.apache.org/bugzilla/show_bug.cgi?id=48841 :
"mod_proxy only allows a BalancerMember's loadfactor to be set to a value
between 1 and 100.  Ideally, loadfactor should be allowed to be set to 0 so
that a BalancerMember can be removed from new traffic but allowed to keep
servicing existing sessions (i.e. bleeding off sessions) or to be able to start
a BalancerMember and test it before letting new traffic access it."

This is also a useful way to gracefully drop a reverse-proxy back-end server from use.  Either one will do.  Would be good as a backport to 2.2 as well.
Comment 7 Keith Mashinter 2011-09-20 01:39:25 UTC
Related to https://issues.apache.org/bugzilla/show_bug.cgi?id=48841 where that allows the magic/special value of lbfactor=0 to support the same idea, taking that server out of the balancer for new sessions but still allowing sticky sessions/routes through.  The other has the advantage of not requiring a new status and reusing the lbfactor, but complicates the lbfactor calculations to ensure they deal with divide-by-zero.
Comment 8 Keith Mashinter 2011-09-20 11:26:03 UTC
Created attachment 27542 [details]
mod_proxy_route_only_for_2_2_21.patch
Comment 9 Keith Mashinter 2011-09-20 11:34:27 UTC
This was resolved in the trunk for 2.4.x as https://issues.apache.org/bugzilla/show_bug.cgi?id=48841.

*** This bug has been marked as a duplicate of bug 48841 ***
Comment 10 Keith Mashinter 2011-09-25 18:03:58 UTC
Created attachment 27585 [details]
Backport of this idea adopted as a Drain status from 2.3/2.4 to Apache-2.2
Comment 11 Keith Mashinter 2011-10-13 02:21:41 UTC
Also note the difference with this and 48841 as compared to the "Route Redirect" functionality is that the "Route Redirect" only redirects existing routes but allows new sessions to be balanced/elected on the server intended to be redirected.

The "Drain" change happens such that ONLY a defined route will make it to the server, and it will never be balanced/elected for requests without its defined route.  This cleanly supports phasing a server completely out of use when all its sessions have ended.

I'll try to keep up with the backport as new 2.2.x versions come out, but it would be nice to include in a futre 2.2.x version.
Comment 12 Keith Mashinter 2012-05-03 04:39:27 UTC
I checked against the Apache 2.2.22 release and the backport of the Drain status from 2.3/2.4 to Apache-2.2 should work for 2.2.22 as well as 2.2.21 since there were no competing changes.
Comment 13 Keith Mashinter 2012-05-12 22:36:19 UTC
Created attachment 28765 [details]
2.2.21 Win32 Release build to add Drain status to proxy balancer workers
Comment 14 Keith Mashinter 2012-05-13 15:07:38 UTC
Created attachment 28769 [details]
2.2.22 Win32 Release build to add Drain status to proxy balancer workers
Comment 15 William A. Rowe Jr. 2012-05-14 18:28:44 UTC
Keith, I am deleting the binary attachments.  We have a quite strict policy of
only posting binaries on apache.org that come from committers to a project.
While we expect anyone applying patches from our issue trackers to carefully
vet and evaluate the suitability of patches for their own compilations, it is
not possible to do this effectively with binary blobs.

Moreso, the bug tracker is not an efficient distribution mechanism, please feel
free to post a link to your own site for users who might want to obtain this
feature in advance of a release, of course the onus is on them and as it isn't
hosted at the ASF, we disclaim any responsibility.

But please don't let this dampen your enthusiam, this is a long sought-after
feature and the contribution was great!
Comment 16 Keith Mashinter 2012-05-14 23:28:18 UTC
Understood that ASF doesn't want the security and other overhead of dealing with binary files in Bugzilla.

I've provided a URL, also below, to download the backport of the proxy balancer/worker Drain status to Apache 2.2.21 and .22.

https://docs.google.com/open?id=0B5zLg35jBZLTMFN3c2JBc014RVU
Comment 17 William A. Rowe Jr. 2012-10-02 20:38:27 UTC
Reopened to tag as fixed.
Comment 18 William A. Rowe Jr. 2012-10-02 20:38:55 UTC
Fixed on trunk and 2.4.x.
Comment 19 Sebb 2012-10-02 22:41:27 UTC
*** Bug 48841 has been marked as a duplicate of this bug. ***
Comment 20 Einar Rosenvinge 2015-04-30 08:15:40 UTC
Does this work?
Three years later, the documentation still states that load factor has range 1..100, and 0 does not work in Apache 2.4.12:

AH00526: Syntax error on line 13 of /.../config/workers.conf:
BalancerMember LoadFactor must be a number between 1..100

Would anyone care to shed some light on this?
Comment 21 Keith Mashinter 2015-05-11 04:15:08 UTC
This was implemented in Apache 2.4 as the Drain status, a separate configuration from the LoadFactor.  I noted a "backport ... to Apache-2.2" as attached, which introduces the Drain status to Apache 2.2 as it would be in 2.4 rather than accepting LoadFactor 0.