[Webtest] Building suites and removing duplication (long post)

Fri, 24 May 2002 09:54:12 +0200

We use an approach similar to the one mentioned by Ben for structuring our
WebTests. Our tests are "use case" oriented in such a way that a test case
represents one path through the use case' activity diagram. Each use case
needs to be executed in the context of a particular business unit so that we
need business unit specific parameter sets for each test case.

It has a directory structure similar to this:

  TEST_HOME/
    config/
       system/
         host1.properties
           (contains properties for a particular target host, e.g.
localhost, test, production, etc.)
         host2.properties
         testenv1.properties
           (contains settings for a particular test environment, e.g.
location of test reports, verbosity level, etc.)
         testenv2.properties
       bu1/
         bu.properties
           (business unit specific properties like, business unit id, logo
image names, links, etc.)
         customer1.properties
           (customer specific properties like customer#, name, etc.)
         customer2.properties
       bu2/
         customer1.properties
         customer2.properties
    include/
      (extracted XML fragments used by all tests, e.g. initial invocation of
application page, etc.)
      XMLFragment1.xml
      XMLFragment2.xml
    scritps/
      (ant scripts used for preparing the system, e.g. setting up the
database)
      SetupDB.xml
    tests/
       AllTests.xml
         (aggregates all AllTests.xml from all functionality groups)
       functionalitygroup1/
         AllTests.xml
           (aggregates all AllTests.xml from all use cases)
	   usecase1.1/
           AllTests.xml
             (invokes each test variation once for all customers and all
business units)
           TestVariation1.xml
             (contains the actual WebTest script with variables (properties)
for business or customer specific properties)
           TestVariation2.xml
	   usecase1.2/
       functionalitygroup2/
         AllTests.xml
	   usecase2.1/
	   usecase2.2/

A TEST_HOME/tests/functionalitygroup1/usecase1.1/AllTests.xml script invokes
the various test case variations for each customer in each business unit
similar to:

  ...
  <target name="main" depend="bu1,bu2" />

  <target name="bu1">
    <property file="${pathToConfig}/bu1/bu.properties" />

    <antcall target="executeTestForAllBu1Customers">
      <param name="testVariationFilename" value="TestVariation1.xml"/>
    </antcall>

    <antcall target="executeTestForAllBu1Customers">
      <param name="testVariationFilename" value="TestVariation2.xml"/>
    </antcall>

  </target>
  ...
  <target name="executeTestForAllBu1Customers">

    <ant antfile="variations/${testVariationFilename}>
      <property file="${pathToConfig}/bu1/customer1.properties"/>
    </ant>

    <ant antfile="variations/${testVariationFilename}">
      <property file="${pathToConfig}/bu1/customer2.properties"/>
    </ant>

  </target>
  ...

A TEST_HOME/tests/functionalitygroup1/AllTests.xml script simply combines
all tests:

   ...
  <target name="main">
    <ant antfile="usecase1.1/AllTests.xml" />
    <ant antfile="usecase1.2/AllTests.xml" />
  </target>
  ...

One could also do a per business unit invocation if desired.

Another approach that we are using is to generate property files and an
AllTests.xml from MS-Excel. The user selects the data sets (i.e. customers)
and test cases. A script will then export the data from an Excel sheet into
a property file and generate the AllTests.xml which invokes the existing
test case variations with the generated property file(s).
This approach has the advantage that administering complex sets of test data
is easier in an application like Excel than in individual property files.

Ciao,
Carsten

Carsten Seibert
seiberTEC GmbH Switzerland
mailto:seibert@seibertec.ch  / phone: +41 79 636 4317

> -----Original Message-----
> From: webtest-admin@lists.canoo.com
> [mailto:webtest-admin@lists.canoo.com]On Behalf Of Ben Cox
> Sent: Freitag, 17. Mai 2002 18:31
> To: webtest@gate.canoo.com
> Subject: Re: [Webtest] Building suites and removing duplication
>
>
> I think this approach is on the right track.  What Tim and I did was
> defined a buildfile called TestSuite.xml at the top-level
> directory of
> our test tree.  That contained the following:
>
> <target name="runTest" depends="initDb">
>    <echo>
>        Executing target: {$targetName} in build file
> ${basedir}/{$filename}
>    </echo>
> </target>
>
> <target name="testSuite" depends="initDb">
>    <ant antfile="admin/AddDataViaAdmin.xml"/>
>    <ant antfile="build/EditToolData.xml"/>
>    ...
> </target>
>
> The initDb target, by the way, is a DBUnit target (Tim and I
> created a
> DBUnit ant task) that populates the database lookup tables.  Then, a
> <dbunit> task is included in most tests before the <testSpec>
> task that
> sets the database to a known state for the test.
> The ant files referred to in the "testSuite" target are all in
> subdirectories, as you can probably tell.  In each of those appears
> something resembling the following:
>
> <target name="testSuite" depends="loginTest, addClient,
> addToolmaker, ...">
>    <echo>Finished testSuite for AddDataViaAdmin.</echo>
> </target>
>
> The ... indicates that all of your targets from that
> buildfile get put
> in there.  Basically, the testSuite for a given file is run with the
> command line:
>
>    ant -buildfile=admin/AddDataViaAdmin.xml testSuite
>
> the testSuite for the whole system is run with the command line:
>
>    ant -buildfile=TestSuite.xml testSuite
>
> and an individual test can be run with a more complicated
> command line
> For example, to run the addClient test target from the
> AddDataViaAdmin.xml file:
>
>    ant -buildfile=TestSuite.xml runtest
> -Dfilename=admin/AddDataViaAdmin.xml -DtargetName=addClient
>
> Though this command line seems a touch unwieldy, in practice
> we found it
> quite usable.
>
> Hope that helps,
>
>    Ben Cox
>
>
>
> Johannes Link wrote:
> > Hi Dierk,
> >
> >
> >>BTW: you don't need to do the entity def in a dtd. You can
> >>extract whole xml nodes in external files.
> >
> >
> > Do you refer to the SYSTEM command in the DTD or something else?
> >
> >
> >>If that does not address your problem:
> >>How would a WebTest look like, where your problem is solved?
> >
> >
> > I could imagine something like:
> > <defstep name="mytask">
> >    <param name="param1"/>
> >    <invoke ...>
> >    <verify ... text=${param1}/>
> > </stepdef>
> >
> > used as:
> > <callstep name="mytask">
> >    <param name="param" value="value1">
> > </callstep>
> >
> > However I'm not sure if this really helps. Alternatively
> one could have
> > another xslt step which transforms our xml (without
> duplication etc.) into a
> > proper webtest xml file. But well, I'm not very
> enthusiastic about this "meta
> > programming with xml/xslt" since it is very difficult not
> to loose control and
> > overview when faced with several transformations in a row.
> >
> > Johannes
> >
>
>
>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest