[Webtest] A failed webest (canoo) should make cruisecontrol send a faiil m essage

Parker, Michael webtest@lists.canoo.com
Thu, 30 Sep 2004 12:40:32 +0200


Hi,

we run also webtests inside cruisecontrol.
I would like to report a failed webtest by a corresponding message of
cruisecontrol.
I found the following posting:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D
Hi David,

> i am using webtest within cruisecontrol and I'd like to make
> cruisecontrol
> build fail if there is any failing webtest step. my build.xml
> tasks are like
> these:
>
>  - check repository for changes
>  - if no changes wait and repeat
>  - checkout project
>  - build and unit testing (with phpunit or junit)
>  - deploy on an integration server
>  - functional testing on the integration server (with webtest)

I guess you have some more steps here, that you don't want to
get executed, if the webtests fail. (e.g. tagging the
current version)

> if everything works fine cruisecontrol sends a "Build Successful"
> email, and
> if something goes wrong it ought send a "Build Failed" message.
> an error in
> junit testing breaks the build and cruisecontrol sends the "Build Failed"
> message, but, and that's my point, an error in webtest testing
> does not break
> the build. why?

By default, WebTests are supposed to work as user acceptance tests
rather than technical verifications. Acceptance tests are likely
not to run 100% from the time they were delivered. Therefore,
WebTest just reports about the progress in terms of running
acceptance tests rather than breaking the build.
This is in contrast to Unit Tests. These are supposed to run
100% with every build.

You can steer this behaviour of WebTest with the
"haltonerror" and "haltonfailure" attributes of the config step.
see http://webtest.canoo.com/webtest/manual/syntax.html#2

For Build number 231 you will find the report of the concerned
selftest step under
http://webtest.canoo.com/selftest/TestResultOverview.html#testspec22

hope that helps and happy testing
Mittie
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
The desciption says:
But haltonerror =3D true will terminate the execution of the ANT script.
Setting the option to "false" will stop the execution of the current
<testSpec> but continues with the next ANT task, e.g. another <testSpec>.
haltonfailure	Defines that the execution of the current test halts if a
failure of one of the test steps is detected, e.g. a <verify...> test did
not pass. Even when set to "false" all of the trailing <step>s of the
current <testSpec> will be skipped and processing will continue with the
next <testSpec>.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
I would like to have the following testpecs to be processed and the
error/failure being reported to cruisencontrol by an ant-property as it is
aleady true for junit-tests like

 <junit haltonfailure=3D"no" fork=3D"yes" failureproperty=3D"junit_test_fai=
led"
printsummary=3D"on">


Does anybody has a hint or a codesnippet for me which shows how it could
work ?

Regards
Michael


_______________________________________________
AXA Service AG
MICHAEL PARKER
PROGRAMMER/ANALYST
Colonia-Allee 10-20; D-51067 K=F6ln
Tel.: +49 (0)221 - 148 - 24 173
Fax:  +49 (0)221 - 148 - 44 - 42 663
E-Mail: mailto:Michael.Parker@axa.de
_______________________________________________



---------------------------------------------------------------------------=
---
Aus Rechts- und Sicherheitsgruenden ist die in dieser E-Mail gegebene Infor=
mation nicht rechtsverbindlich. Eine rechtsverbindliche Bestaetigung reiche=
n wir Ihnen gerne auf Anforderung in schriftlicher Form nach. Beachten Sie =
bitte, dass jede Form der unautorisierten Nutzung, Veroeffentlichung, Vervi=
elfaeltigung oder Weitergabe des Inhalts dieser E-Mail nicht gestattet ist.=
 Diese Nachricht  ist ausschliesslich fuer den bezeichneten Adressaten oder=
 dessen Vertreter bestimmt. Sollten Sie nicht der vorgesehene Adressat dies=
er E-Mail oder dessen Vertreter sein, so bitten wir Sie, sich mit dem Absen=
der der E-Mail in Verbindung zu setzen.
----------------------------
For legal and security reasons the information provided in this e-mail is n=
ot legally binding. Upon request we would be pleased to provide you with a =
legally binding confirmation in written form. Any form of unauthorised use,=
 publication, reproduction, copying or disclosure of the content of this e-=
mail is not permitted. This message is exclusively for the person addressed=
 or their representative. If you are not the intended recipient of this mes=
sage and its contents, please notify the sender immediately.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D