[Webtest] ant/webtest accumulating ram space and stalling

Rob Manthey webtest@lists.canoo.com
Thu, 22 Jun 2006 18:15:38 +1000

(I drafted this a few months ago and then shelved it as I couldn't 
verify that I hadn't caused my own environment problem, but Marc's 
recent mail about memory leaks reminded me of it, so I'll post it in 
case it holds some value for anyone.  I still have the problem ... )

First post to this list, so Hi to all!

My problem is probably a fairly simple config oversight and probably not 
even a webtest issue in fact, but I'm stuck so here goes:
I added some more tests on my 
Tomcat5.5/Struts1.2/Webtest1.7/Java5.0/RHEL4 web app recently and my ant 
driven webtests are now grinding to a halt.  Individually each test runs 
fine and releases the used ram (viewed via linux system monitor gui), 
but I have a master ant build.xml to run them all and as it runs 
sequential tests ram is not released and it gradually grinds to a halt 
and the tests just plain stop mid-run.  The linux "system monitor" graph 
of "memory used" rises, plateaus and the twin cpus show one at 100% and 
the other at 5%, swapping every few minutes, so 1 cpu is blocked (may 
not be the correct use of the term?).  I've left it half an hour and it 
doesn't seem to proceed any furher.

I'm testing a jsp/servlet/tomcat based app that has a fairly off-beat 
MVC model and session management segment (it's not reliant on the j2ee / 
http session scoping, so that I can maintain concurrent individual 
window sessions in any browser and share data between user logins).  
Hence I may not be releasing some session objects from scope as the http 
call exits (I have put some work into getting this right, but could do 
more I guess).

In the linux system monitor gui the graphic view of "Used Memory" climbs 
steadily by some 200M as the script runs until it stalls (well short of 
the 1G limit and no disk caching is occurring; no change in behaviour 
either if I shut down other big-memory programs).  On the other hand, 
the ram used by the individual processes in question stays near enough 
to the same (Tomcat's and the ant script's usage climb by a few Meg then 
drop back as sessions log on and then close, which is what I'd expect 
for what I'm doing).  Once it stalls, if I kill ant's java process the 
lost ram returns (the graph drops back by 200M) and I can re-run it with 
the same behaviour.

Any thoughts?  JVM out of memory was my nagging thought, but I have 
increased the ram allocations both for ant and for the jvm via ANT_OPTS 
and JAVA_OPTS in the .bashrc file and the ram allocation increases by 
that amount in the "System Monitor - Process Listing - Memory", but the 
task still grinds to a halt at the same step in ant.  Perhaps the memory 
allocation for the jvm that ant is actually using (perhaps not the 
.bashrc targetted one) is being set somewhere that I haven't found?

Thanks for reading and also for any time spent thinking about it for 
me.  I'm hoping someone has some info or at least some "try/read this" 
leads for me to follow up next.