[Webtest] Resume after failing step

Michal Michal <uhmm@1234.sk>
Thu, 12 Apr 2007 10:16:13 +0200


when I started to work with webtest I also missed extended haltonfailure 
For example in our tests after we load page we do common verification 
for valid urls, 404 errors, we load css, images, etc...
If this test fails it is not critical to whole <webtest> section, we 
would like to know about the error but at the same time we would like to 
continue testing.

Therefore it would be excellent if we would be able to create a step 
like <verifyTitle text="my title" onfailure="continuetests" /> which 
would indicate that this step is not critical for the whole webtest and 
steps should be executed as if <verifyTitle> passed even if it failed.

What do you think?

Marc Guillemot wrote:
> haltonfailure and haltonerror both allow to configure if the build
> should fail or not. Ie if Ant should stop the execution after </webtest>.
> In both cases the execution of the steps is stopped after the failing step.
> Generally I think quite strange to continue a test when something
> failed. How to consider the results at the end? Correct or not correct?
> Nevertheless it may make sense in some cases. Denis mentioned a case to
> me last week where he had some <verifyXxxx/> to check the Italian
> translation of some text. The application behaved correctly but some of
> the texts where wrong and it would have been more useful to complete the
> execution and show all wrong texts in one time.
> @Christoph
> is your case comparable with Denis' one?
> Perhaps should you refactor your tests. If you have a long way to go to
> a particular page and want to perform different actions from there, what
> about splitting it? A first test would just go to the particular page
> and store on file session information (probably the cookies that allow
> to hold the session). Then each single "test from the particular page"
> could just reuse this session by setting the cookie value from what is
> read from file. This would give you a far better information in report:
> you would see precisely which single "test from the particular page"
> works and which fails.
> Marc.
>  seems the same to me as
>> if it were true or false.
>> If it is false, next steps got skipped
>> If it is true, next steps got skipped anyways, so I don't really see the
>> difference
>> Christoph Sandrus wrote:
>>> On 4/11/07, *Michael Habbert* <Michael.Habbert@netpioneer.de
>>> <mailto:Michael.Habbert@netpioneer.de>> wrote:
>>>     Hi Christoph,
>>>     Christoph Sandrus wrote:
>>>     > Hi
>>>     >
>>>     [...]
>>>     >      >haltonfailure
>>>     >      >    Required? No, default is "true"
>>>     >
>>>     >     http://webtest.canoo.com/webtest/manual/config.html
>>>     >
>>>     >
>>>     > Yes I know this feature, but if a failure occures, the trailing
>>>     steps
>>>     > (and groups) are skipped. And webtest goes on with the next webtest
>>>     > section. (So I have to login and do the same stuff, I have
>>>     already done,
>>>     > again)
>>>     [...]
>>>     you do know the diference between haltonfailure and haltonerror?
>>> If I write haltonfailure="false" the trailing steps are skipped. So I
>>> didn't get the point here?
>>> Regards,
>>> Christoph
>> _______________________________________________
>> WebTest mailing list
>> WebTest@lists.canoo.com
>> http://lists.canoo.com/mailman/listinfo/webtest
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest