[Webtest] Re: Data driven tests

Jason Kissinger webtest@lists.canoo.com
Fri, 23 Jul 2004 14:28:32 -0500


I didn't intend this to be merely a csv file load (similar to the
Property task), but more like a For task (see=20
http://ant-contrib.sourceforge.net/tasks/for.html).  It's similar to a
for loop in most programming languages.  For each line of the csv file,
it calls the nested tasks.  Prior to beginning each iteration, it loads
the current row from the csv file into properties pointed to by the
'param' attribute.  These properties include param.columnN for each
column value in the current row, and param.rowNum for the current row
being run.  Other properties could be set pertaining to the state of the
csv data, but these seem the most important to me.

For me, this seems to be all that is needed to do data driven testing
with webtest.  Read in a set of inputs (with say columns 1-5) and verify
the content of resulting page(s) with other columns (say 6-10).  That
way our script developers can write the scripts, while the business
analysts add test cases via the csv in OpenOffice/Excel.

<echo>running...</echo>
<csvFile file=3D"data.csv" param=3D"foo">
   <echo>Currently working on row ${foo.rowNum}</echo>
   <echo>  first column:  ${foo.column1}</echo>
   <echo>  second column:  ${foo.column2}</echo>
   <echo/>
</csvFile>
<echo>done.</echo>

with data.csv:
1,2
2,3
asdf,fdsa

would run as:
running...
Currently working on row 1
  first column:  1
  second column:  2

Currently working on row 2
  first column:  2
  second column:  3

Currently working on row 3
  first column:  asdf
  second column:  fdsa

done.



On Fri, 2004-07-23 at 12:55, Dierk Koenig wrote:
> Hi,
>=20
> I really like the idea of putting properties in tables.
>=20
> I'd like to propose some modification of the csvData step
> design:
>=20
> 1) I'd prefer
> <steps>
>   <csvData file=3D"data/reports.csv" param=3D"csv" />
>   <invoke stepid=3D"1.0 get reports page" .../>
> 	...
> </steps>
> over the "nested" solution.
> It is much less effort in implementation, testing and
> especially for proper reporting.

>=20
> 2) What is the attribute param=3D"csv" for?
> The stepname already supposes that only csv is supported.
>=20
> 3) My typical use of property-tables is to refer to entries
> by row-number and column-name or even by "primary-key" and
> column name, fairly much like the "vlookup" feature in Excel.
> How about this:
>=20
> <csvData file=3D"data/sample.csv" id=3D"sample" />
> <invoke stepid=3D"#{sample.0.bye}" .../> <!-- by index, evaluates to
> "Tsch=FCss" -->
> <invoke stepid=3D"#{sample.eng.greet}" .../> <!-- by vlookup, evaluates t=
o
> "Hi" -->
>=20
> Where data file is:
> lan	greet	bye
> ger	Hallo	Tsch=FCss
> eng	Hi	Goodbye
>=20
> (first line contains headers, first row is used for vlookup)
>=20
> 4) as in the example above it seems to me that using
> dynamic properties (i.e. the #{} notation rather than the ${}
> notation) is more approriate. Otherwise you could implement
> a plain ant task for this rather than a webtest step.
>=20
> N.B.: neither #{} nor ${} allow for nested resolution like
> one would need inside the repeat step:
> <repeat stepid=3D"does not work" count=3D"2">
> 	<invoke stepid=3D"#{sample.#{count}.C}" .../>
> </repeat>
> This limits the usability of reference-by-index.
>=20
> What to do about this?
> Well, we have gone a totally different approach.
> First, we split into test execution targets and
> test suites, e.g.
> <target name=3D"example.execution">
> ...
> 	<invoke stepid=3D"${greet}" .../>
> 	<invoke stepid=3D"${bye}" .../>
> ...
> </target>
>=20
> and have a separate suite file containing
> calls like
> <target name=3D"english"
> 	<ant file=3D"execution.xml" target=3D"example.execution">
> 		<property name=3D"greet" value=3D"Hi"/>
> 		<property name=3D"bye" value=3D"Goodbye"/>
> 	</ant>
> </target>
> (If test data is organized in property files, we only use
> the nested step <property file=3D"english.properties"/> but
> we want to go for tables, right?)
>=20
> Now the trick:
> It is very easy to _generate_ this suite on the fly since
> they contain the very same data like the table.
> We have done this directly from excel-files via a Ruby script.
> I admit, this can be done in VBA but then it is
> tricky to unit-test the generation :-)
> Xls is a nice format for playing with the data but
> not so good for versioning.
>=20
> cheers
> Mittie
>=20
> -----Original Message-----
> From: webtest-admin@lists.canoo.com
> [mailto:webtest-admin@lists.canoo.com]On Behalf Of Jason Kissinger
> Sent: Freitag, 23. Juli 2004 17:00
> To: webtest@lists.canoo.com
> Subject: [Webtest] Re: Data driven tests
>=20
>=20
> I've also started working on a csv loop for webtest similar to the
> foreach task from ant-contrib.  My implementation adds a csvData task
> similar to what you are proposing.  I should have this ready to share
> early next week if anyone is interested.
>=20
> <steps>
>   <csvData file=3D"data/reports.csv" param=3D"csv">
>     <invoke stepid=3D"1.0 get reports page"
>         url=3D"path/to/reports.jsp"
>         username=3D"${csv.column1}"
>         password=3D"${csv.column2}"/>
>     <setselectfield stepid=3D"1.1 select report type: ${csv.column3}"
>         name=3D"rptType"
>         text=3D"${csv.column3}"/>
>     <setinputfield stepid=3D"1.2 set date: ${csv.column4}"
>         name=3D"effDate"
>         text=3D"${csv.column4}"/>
>     <clickbutton stepid=3D"1.3 submit report request"
>         label=3D"Generate Report"/>
>     <verifytext stepid=3D"2.0 verify report result: ${csv.column5}"
>         text=3D"${csv.column5}"
>         regex=3D"true"/>
>   </csvData>
> </steps>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>=20
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest