[Webtest] 32000 tests

[testing Auto]

I automated the creation of web tests.  Basically I
saw a pattern in the 24 or so WebTests that I'd
written using "by hand."  As I explained to my boss
what I was doing he of course pointed out what the key
to the pattern was and w/ a week or so of work I
integrated my solution w/ webtest.  Now I have 32,000
possible tests from one template.  I have about 4
other templates that have <32,000 possiblities, hence
the 32,000+.  

As for impact.  Well, the time bites!!! The one
template takes 3 hours.  The others vary between 30
seconds to 20 minutes depending on how many of the
32,000 possible patterns are used.  As for impact on
the project it hasn't been large yet.  The
configuration manager (new) is still working to
automate the build.  He has not been converted to ANT!
 And he is not thrilled about the time, neither am I,
but currently it takes the testing crew 2 weeks to do
a full regression test.  While my little monster
doesn't do all of those tests, it does quite a few,
and in some areas more than what they are doing.  

As for maintenance, I'm sure you were thinking that I
had 1000's of xml files w/ these tests in them, but
since its about 700 lines of code spilt between 4 or 5
classes its really not difficult.  In fact I'm hoping
to add a few more features, based on the customer's
needs, or course.  If I can get pass the HTTPS and
Authentication for the beta they will be happy.

However, next they want a WebPage where they type in
the info for the template and it will get added to the
WebTest suite of tests and when the next build runs
the new/old will all be executed.  Hopefully in this
way we will be able to turn as many new bug reports
into new automated tests as possible.

Chris

PS.  If you want to know more details I'd love to
share, but since I made the simplest possible solution
that could possibly work, for my problem it my not be
to applicable.

PPS.  This was my first attempt on the web tier, on
the database tier I have 3 groups of test that
generate 5 million+ tests.  This takes 3 days to run
and currently there are only 7,000 failures.  (That's
6990 more than what they knew about before I wrote the
tests engine).  

PPPS.  Oh one more thing.  All of my tests are
considered "Acceptance" tests.  I'm verifing what we
as the customer asked the contractor to develop.


   Dierk_Konig  wrote: Hi Chris, your posting made me
curious.Can you tell a little more about the 32,000+
tests?How did you achieve such a large number? How
long does ist take to run them? How is maintenance?How
is the effect on the project / developers / customers
/ managers?cheersMittie-----

__________________________________________________
Do You Yahoo!?
Yahoo! Health - your guide to health and wellness
http://health.yahoo.com