[Webtest] Re: Data driven tests

Dierk Koenig webtest@lists.canoo.com
Fri, 23 Jul 2004 19:55:03 +0200


Hi,

I really like the idea of putting properties in tables.

I'd like to propose some modification of the csvData step
design:

1) I'd prefer
<steps>
  <csvData file="data/reports.csv" param="csv" />
  <invoke stepid="1.0 get reports page" .../>
	...
</steps>
over the "nested" solution.
It is much less effort in implementation, testing and
especially for proper reporting.

2) What is the attribute param="csv" for?
The stepname already supposes that only csv is supported.

3) My typical use of property-tables is to refer to entries
by row-number and column-name or even by "primary-key" and
column name, fairly much like the "vlookup" feature in Excel.
How about this:

<csvData file="data/sample.csv" id="sample" />
<invoke stepid="#{sample.0.bye}" .../> <!-- by index, evaluates to
"Tschüss" -->
<invoke stepid="#{sample.eng.greet}" .../> <!-- by vlookup, evaluates to
"Hi" -->

Where data file is:
lan	greet	bye
ger	Hallo	Tschüss
eng	Hi	Goodbye

(first line contains headers, first row is used for vlookup)

4) as in the example above it seems to me that using
dynamic properties (i.e. the #{} notation rather than the ${}
notation) is more approriate. Otherwise you could implement
a plain ant task for this rather than a webtest step.

N.B.: neither #{} nor ${} allow for nested resolution like
one would need inside the repeat step:
<repeat stepid="does not work" count="2">
	<invoke stepid="#{sample.#{count}.C}" .../>
</repeat>
This limits the usability of reference-by-index.

What to do about this?
Well, we have gone a totally different approach.
First, we split into test execution targets and
test suites, e.g.
<target name="example.execution">
...
	<invoke stepid="${greet}" .../>
	<invoke stepid="${bye}" .../>
...
</target>

and have a separate suite file containing
calls like
<target name="english"
	<ant file="execution.xml" target="example.execution">
		<property name="greet" value="Hi"/>
		<property name="bye" value="Goodbye"/>
	</ant>
</target>
(If test data is organized in property files, we only use
the nested step <property file="english.properties"/> but
we want to go for tables, right?)

Now the trick:
It is very easy to _generate_ this suite on the fly since
they contain the very same data like the table.
We have done this directly from excel-files via a Ruby script.
I admit, this can be done in VBA but then it is
tricky to unit-test the generation :-)
Xls is a nice format for playing with the data but
not so good for versioning.

cheers
Mittie

-----Original Message-----
From: webtest-admin@lists.canoo.com
[mailto:webtest-admin@lists.canoo.com]On Behalf Of Jason Kissinger
Sent: Freitag, 23. Juli 2004 17:00
To: webtest@lists.canoo.com
Subject: [Webtest] Re: Data driven tests


I've also started working on a csv loop for webtest similar to the
foreach task from ant-contrib.  My implementation adds a csvData task
similar to what you are proposing.  I should have this ready to share
early next week if anyone is interested.

<steps>
  <csvData file="data/reports.csv" param="csv">
    <invoke stepid="1.0 get reports page"
        url="path/to/reports.jsp"
        username="${csv.column1}"
        password="${csv.column2}"/>
    <setselectfield stepid="1.1 select report type: ${csv.column3}"
        name="rptType"
        text="${csv.column3}"/>
    <setinputfield stepid="1.2 set date: ${csv.column4}"
        name="effDate"
        text="${csv.column4}"/>
    <clickbutton stepid="1.3 submit report request"
        label="Generate Report"/>
    <verifytext stepid="2.0 verify report result: ${csv.column5}"
        text="${csv.column5}"
        regex="true"/>
  </csvData>
</steps>
_______________________________________________
WebTest mailing list
WebTest@lists.canoo.com
http://lists.canoo.com/mailman/listinfo/webtest