[Webtest] Why do you use WebTest? ;-)

Lisa Crispin Lisa Crispin" <lisa.crispin@gmail.com
Fri, 24 Nov 2006 13:19:16 -0700


------=_Part_48562_436071.1164399556684
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

We use a combination of tools for test automation.  We could have purchased
vendor tools, we chose the tools we have because they fit our needs.

The tools we use are:
JUnit for programmer-facing tests (we practice TDD)
FitNesse for customer-facing tests behind the GUI (and occasionally,
programmer-facing tests, when it's easier to do TDD for a particular bit of
code that way)
WebTest for regression tests using the GUI
WATIR for some GUI regression testing and scripts to set up for manual
tests, where we needed a lot of flexibility and particular features (WebTest
probably provides some of the features through using Groovy or similar, but
for this it was easier for us to ramp up on Ruby)

We chose WebTest 3 years ago when we started an agile development approach.
We had no automated tests at all.  We looked for a tool that was
programmer-friendly, but easy for a tester with some technical skills to
ramp up on, fairly independently.  The fact that it integrates so easily
with CruiseControl was another big 'selling' point.  Within 8 months, we had
a suite of smoke tests than covered all the critical parts of our app.  This
suite first ran as part of a nightly build, now it runs as part of a
continual 'full' build that also runs FitNesse suites.

At some point we had a lot of trouble with Javascript so we started using
WATIR for scripts where we couldn't get WebTest to work.  I think this was a
couple years ago.  My fellow tester was a Perl programmer so it was easy for
him to get up to speed on Ruby and he wrote some nice scripts that accepted
lots of command line parameters to set up particular data scenarios for
manual testing.  He preferred WATIR to WebTest, just a personal preference,
so he tended to automate all his GUI tests there.  We still used our
extensive WebTest suite.  The WATIR tests are very difficult and expensive
to maintain.  The tests have to be tested because they contain complex
logic, whereas the WebTest scripts are specified and we rarely use any logic
such as the ifStep.  Also, we haven't gone to the trouble to try to
integrate WATIR scripts with our build process (since it runs on Windows).

At Agile 2006 I attended Paul King's WebTest tutorial and realized there
were lots of robust new features.  Meanwhile, the author of most of our
WATIR scripts left and the maintenance  had become a big burden.  Everyone
was having to learn Ruby (I enjoyed learning it myself, but it is difficult
for me) and try to understand some complex code in order to work on the
WATIR scripts.

We looked back to WebTest and discovered all the Javascript issues we had
with WebTest scripts in the past were solved, and we've been automating
pretty much all of our GUI tests in WebTest since then.  The tests are much
quicker to write and cheap to maintain.  We have to sharpen up some of our
HTML coding but we should do that anyway.  We will keep our WATIR script and
maintain it, but I expect we will write most tests for new features in
WebTest.

We are very happy with our combination of tools covering our different
needs.  We're always open to new tools - a couple of the programmers saw
Selenium at a conference and are eager to explore it more.  But as the
changes in WebTest have shown, good tools grow and change, and the team's
needs or skill sets may change too, so it's important to always keep all
options open.

Our WebTest scripts, while quite simple, find regression bugs quite often.
We don't have to change them often, and when we do (due to UI changes) it
goes quickly.  The ROI on them is tremendous.  I've used many vendor tools
successfully over the past 10 - 12 years, and the ROI on WebTest is many
orders of magnitude higher than any tool I've used.  Our programmers are
able and willing to update scripts when necessary (with much less
complaining than they do about the Ruby scripts) and our entire team is
happy with it.
-- Lisa


On 11/23/06, Marc Guillemot <mguillemot@yahoo.fr> wrote:
>
> Hi all,
>
> why do you use WebTest rather than some other test tool (commercial or
> open source)?
>
> Of course I have some ideas on this question ;-) but as WebTest
> committer I'm surely not objective when comparing WebTest to others.
> While presenting WebTest I've already encountered difficulties to
> convince that it was the tool of choice. I think that it occurs
> particularly when people don't have any real experience of automated web
> application tests and therefore can't understand "real" arguments. From
> your experience as WebTest user, what would you say to persuade someone
> wanting to start test automation for a webapp to use WebTest rather than
> something else?
>
> Marc.
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>



-- 
Lisa Crispin
Co-author,
Testing Extreme Programming
http://lisa.crispin.home.att.net

------=_Part_48562_436071.1164399556684
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

We use a combination of tools for test automation.&nbsp; We could have purchased vendor tools, we chose the tools we have because they fit our needs.<br><br>The tools we use are:<br>JUnit for programmer-facing tests (we practice TDD)
<br>FitNesse for customer-facing tests behind the GUI (and occasionally, programmer-facing tests, when it's easier to do TDD for a particular bit of code that way)<br>WebTest for regression tests using the GUI<br>WATIR for some GUI regression testing and scripts to set up for manual tests, where we needed a lot of flexibility and particular features (WebTest probably provides some of the features through using Groovy or similar, but for this it was easier for us to ramp up on Ruby)
<br><br>We chose WebTest 3 years ago when we started an agile development approach.&nbsp; We had no automated tests at all.&nbsp; We looked for a tool that was programmer-friendly, but easy for a tester with some technical skills to ramp up on, fairly independently.&nbsp; The fact that it integrates so easily with CruiseControl was another big 'selling' point.&nbsp; Within 8 months, we had a suite of smoke tests than covered all the critical parts of our app.&nbsp; This suite first ran as part of a nightly build, now it runs as part of a continual 'full' build that also runs FitNesse suites.
<br><br>At some point we had a lot of trouble with Javascript so we started using WATIR for scripts where we couldn't get WebTest to work.&nbsp; I think this was a couple years ago.&nbsp; My fellow tester was a Perl programmer so it was easy for him to get up to speed on Ruby and he wrote some nice scripts that accepted lots of command line parameters to set up particular data scenarios for manual testing.&nbsp; He preferred WATIR to WebTest, just a personal preference, so he tended to automate all his GUI tests there.&nbsp; We still used our extensive WebTest suite.&nbsp; The WATIR tests are very difficult and expensive to maintain.&nbsp; The tests have to be tested because they contain complex logic, whereas the WebTest scripts are specified and we rarely use any logic such as the ifStep.&nbsp; Also, we haven't gone to the trouble to try to integrate WATIR scripts with our build process (since it runs on Windows).&nbsp; 
<br><br>At Agile 2006 I attended Paul King's WebTest tutorial and realized there were lots of robust new features.&nbsp; Meanwhile, the author of most of our WATIR scripts left and the maintenance&nbsp; had become a big burden.&nbsp; Everyone was having to learn Ruby (I enjoyed learning it myself, but it is difficult for me) and try to understand some complex code in order to work on the WATIR scripts. 
<br><br>We looked back to WebTest and discovered all the Javascript issues we had with WebTest scripts in the past were solved, and we've been automating pretty much all of our GUI tests in WebTest since then.&nbsp; The tests are much quicker to write and cheap to maintain.&nbsp; We have to sharpen up some of our HTML coding but we should do that anyway.&nbsp; We will keep our WATIR script and maintain it, but I expect we will write most tests for new features in WebTest.&nbsp; 
<br><br>We are very happy with our combination of tools covering our different needs.&nbsp; We're always open to new tools - a couple of the programmers saw Selenium at a conference and are eager to explore it more.&nbsp; But as the changes in WebTest have shown, good tools grow and change, and the team's needs or skill sets may change too, so it's important to always keep all options open.
<br><br>Our WebTest scripts, while quite simple, find regression bugs quite often.&nbsp; We don't have to change them often, and when we do (due to UI changes) it goes quickly.&nbsp; The ROI on them is tremendous.&nbsp; I've used many vendor tools successfully over the past 10 - 12 years, and the ROI on WebTest is many orders of magnitude higher than any tool I've used.&nbsp; Our programmers are able and willing to update scripts when necessary (with much less complaining than they do about the Ruby scripts) and our entire team is happy with it.
<br>-- Lisa<br><br><br><div><span class="gmail_quote">On 11/23/06, <b class="gmail_sendername">Marc Guillemot</b> &lt;<a href="mailto:mguillemot@yahoo.fr">mguillemot@yahoo.fr</a>&gt; wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi all,<br><br>why do you use WebTest rather than some other test tool (commercial or<br>open source)?<br><br>Of course I have some ideas on this question ;-) but as WebTest<br>committer I'm surely not objective when comparing WebTest to others.
<br>While presenting WebTest I've already encountered difficulties to<br>convince that it was the tool of choice. I think that it occurs<br>particularly when people don't have any real experience of automated web<br>application tests and therefore can't understand &quot;real&quot; arguments. From
<br>your experience as WebTest user, what would you say to persuade someone<br>wanting to start test automation for a webapp to use WebTest rather than<br>something else?<br><br>Marc.<br>_______________________________________________
<br>WebTest mailing list<br><a href="mailto:WebTest@lists.canoo.com">WebTest@lists.canoo.com</a><br><a href="http://lists.canoo.com/mailman/listinfo/webtest">http://lists.canoo.com/mailman/listinfo/webtest</a><br></blockquote>
</div><br><br clear="all"><br>-- <br>Lisa Crispin<br>Co-author, <br>Testing Extreme Programming<br><a href="http://lisa.crispin.home.att.net">http://lisa.crispin.home.att.net</a>

------=_Part_48562_436071.1164399556684--