[Webtest] Why do you use WebTest? ;-)

John and Pip John and Pip" <pipandjohn@gmail.com
Fri, 24 Nov 2006 07:55:32 +1300


------=_Part_87349_7908131.1164308132271
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi Marc,

Jury is still out as to whether we will actually end up using it, but the
reasons it is one of two I'm evaluting is:

- Ease of install, upgrade, maintenance compared to the Big4 tools such as
Rational.
- Cost = $0
- Support = good and $0
- Advantages of declarative XML format but with addition of good scripting
support (that would probably let us code the action-word framework in Groovy
if we needed to - see below)
- Open architecture
- Support parameters to suite e.g. environment or base URL as well as to
test case and step.
- Support nested tests and suites
- Support compex named data elements e.g. person that can be retrieved from
separate "file" to the test steps and simple data elements.

Areas where it is not bad but perhaps some others are better are:

- Handle IE and Firefox. WebTest does not drive a real browser. We need
pages that work in IE and Firefox so ideally in our web-gui-based testing we
would be testing this, not functionality in the simulated environment of
HTMLUnit. One example of this is having to put ? in path of URL instead of /
to avoid 302 errors that I posted about last week. Another example is the
confirmation page not being found that I posted about this week.
- Handling of JavaScript. Problem here is if we have a page that has
javascript that makes HTMLUnit spew, but that page uses a javascript link to
move to next page in flow, my end to end test dies
- Can be set to log and to produce screenshots esp. on error. This sort of
works I guess but without the stylesheet references.
- Must not break if page contains ActiveX, Flash, .Net constructs and
whatever else. I need a tool that doesn't force me to ask developers to
change code.
- Handles AJAX
- Can read input files and produce output files in one of and ideally all of
csv, Excel, XML. This can probably be handled by Groovy where necessary.
- No duplication so each part of system (usually each screen) needs just one
fixture accessing it. Specifically we have been looking at keyword-driven,
also called action-word, frameworks, and I'm trying to figure out how the
rather different paradigm of WebTest maps to this, and reviewing in fact
whether our asumptions that action-word is the way to go is correct. Ideally
want just one "script" having to understand the implementation details of
each page or chunk of a page.

For the record, the comparison I'm using in terms of more traditional tools
is QEngine from AdventNet. It is not free but at $700 US is a lot cheaper
than the Big4 and has excellent support.

I think WebTest is a great tool, and the developers have done a wonderful
job. I think WebTest and Fit/Fitnesse represent the future of web gui
functional testing. If WebTest could transparently select whether to use
HTMLUnit or a real browser to execute the same test (not sure how - don't
like selenium etc that require proxies so would I think be best to drive
browser through their own native interface e.g. Com/ActiveX I guess for IE),
and if it could be supported as a fixture within Fitnesse, that would be
Nirvana for me.

regards,
       John

On 11/24/06, Marc Guillemot <mguillemot@yahoo.fr> wrote:
>
> Hi all,
>
> why do you use WebTest rather than some other test tool (commercial or
> open source)?
>
> Of course I have some ideas on this question ;-) but as WebTest
> committer I'm surely not objective when comparing WebTest to others.
> While presenting WebTest I've already encountered difficulties to
> convince that it was the tool of choice. I think that it occurs
> particularly when people don't have any real experience of automated web
> application tests and therefore can't understand "real" arguments. From
> your experience as WebTest user, what would you say to persuade someone
> wanting to start test automation for a webapp to use WebTest rather than
> something else?
>
> Marc.
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>

------=_Part_87349_7908131.1164308132271
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi Marc,<br><br>Jury is still out as to whether we will actually end up using it, but the reasons it is one of two I'm evaluting is:<br><br>- Ease of install, upgrade, maintenance compared to the Big4 tools such as Rational.
<br>- Cost = $0<br>- Support = good and $0<br>- Advantages of declarative XML format but with addition of good scripting support (that would probably let us code the action-word framework in Groovy if we needed to - see below)
<br>- Open architecture<br>- Support parameters to suite e.g. environment or base URL as well as to test case and step.<br>- Support nested tests and suites<br>- Support compex named data elements e.g. person that can be
retrieved from separate &quot;file&quot; to the test steps and simple data
elements.&nbsp;

<br><br>Areas where it is not bad but perhaps some others are better are:<br><br>- Handle IE and Firefox. WebTest does not drive a real browser. We need pages that work in IE and Firefox so ideally in our web-gui-based testing we would be testing this, not functionality in the simulated environment of HTMLUnit. One example of this is having to put ? in path of URL instead of / to avoid 302 errors that I posted about last week. Another example is the confirmation page not being found that I posted about this week.
<br>- Handling of JavaScript. Problem here is if we have a page that has javascript that makes HTMLUnit spew, but that page uses a javascript link to move to next page in flow, my end to end test dies<br>- Can be set to log and to produce screenshots esp. on error. This sort of works I guess but without the stylesheet references.
<br>- Must not break if page contains ActiveX, Flash, .Net constructs and whatever else. I need a tool that doesn't force me to ask developers to change code.<br>- Handles AJAX<br>- Can read input files and produce output files in one of and ideally all of csv, Excel, XML. This can probably be handled by Groovy where necessary.
<br>- No duplication so each part of system (usually each screen) needs
just one fixture accessing it. Specifically we have been looking at
keyword-driven, also called action-word, frameworks, and I'm trying to
figure out how the rather different paradigm of WebTest maps to this,
and reviewing in fact whether our asumptions that action-word is the
way to go is correct. Ideally want just one &quot;script&quot; having to
understand the implementation details of each page or chunk of a page.<br><br>For the record, the comparison I'm using in terms of more traditional tools is QEngine from AdventNet. It is not free but at $700 US is a lot cheaper than the Big4 and has excellent support.
<br><br>I think WebTest is a great tool, and the developers have done a wonderful job. I think WebTest and Fit/Fitnesse represent the future of web gui functional testing. If WebTest could transparently select whether to use HTMLUnit or a real browser to execute the same test (not sure how - don't like selenium etc that require proxies so would I think be best to drive browser through their own native interface 
e.g. Com/ActiveX I guess for IE), and if it could be supported as a fixture within Fitnesse, that would be Nirvana for me.<br><br>regards,<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; John<br>
<br><div><span class="gmail_quote">On 11/24/06, <b class="gmail_sendername">Marc Guillemot</b> &lt;<a href="mailto:mguillemot@yahoo.fr">mguillemot@yahoo.fr</a>&gt; wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi all,<br><br>why do you use WebTest rather than some other test tool (commercial or<br>open source)?<br><br>Of course I have some ideas on this question ;-) but as WebTest<br>committer I'm surely not objective when comparing WebTest to others.
<br>While presenting WebTest I've already encountered difficulties to<br>convince that it was the tool of choice. I think that it occurs<br>particularly when people don't have any real experience of automated web<br>application tests and therefore can't understand &quot;real&quot; arguments. From
<br>your experience as WebTest user, what would you say to persuade someone<br>wanting to start test automation for a webapp to use WebTest rather than<br>something else?<br><br>Marc.<br>_______________________________________________
<br>WebTest mailing list<br><a href="mailto:WebTest@lists.canoo.com">WebTest@lists.canoo.com</a><br><a href="http://lists.canoo.com/mailman/listinfo/webtest">http://lists.canoo.com/mailman/listinfo/webtest</a><br></blockquote>
</div><br>

------=_Part_87349_7908131.1164308132271--