Comment from the dev list:
Was this test/work run/done against trunk, or against 0.8?
The 0.8 release (and to much lesser extent 0.6 too) had several extremely nasty memory issues, but whilst undertaking the work for 0.10 that Andrew referenced below I was able to run a couple thousand connections into the broker within around 20MB of heap at the end (on a 64bit JVM, so knock maybe 75% off for a 32bit JVM). I have just performed the indicated test and had the same result.
Having taken and examined the heap dump of the above test I can see that we are indeed leaking connections and associated objects, however not for the reasons indicated so far. The 0-10 connections historically were not added to the ConnectionRegistry but now are being added, and the code for the feature which added them also does remove them when that feature is used, however it fails to remove them when the close is invoked from the client side (ie, the normal case). As a result, the connection is left in the ConnectionRegistry and is held in memory along with its associated Objects. (...and I have jsut noticed Andrew said as much below, doh..always read the full mail )
I dont think nulling the references between the objects in question is the way to go here, it might help prevent the impact to a certain extent but it doesnt fix the underlying issue and may just introduce more (there is almost certainly scope for NPEs in there).
Having crudely put in removal of the connections from the registry upon a standard clsoe, I was able to run 2000 connections into the broker and come out with 4MB of heap used instead of 13MB previously. I will look to properly put this fix into 0.11/0.12 soon, but its too late for 0.10 now; since this is actually not a regression since 0.6 or 0.8 (from which memory usage is actually massively improved) I definitely dont think its a blocker. As Andrew also noted, a specific test for this might be useful.