[Webtest] Groovy Integration Questions

Florent Blondeau Florent Blondeau <fblondeau@pingwy.com>
Wed, 02 Jul 2008 12:50:27 +0200


This is a multi-part message in MIME format.
--------------000705050207090502050001
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit

Hi,

I take the time to give my point of view, which is not the best one but 
some ideas could interest you.
I've built an extensible test suite entirely in Groovy and found it 
astonishing.
The based examples are documented, and that's all you need to begin 
with. Well, it's just a few lines of code to write in Eclipse
or your favorite IDE  (here is an example of how to validate a page 
against W3C):
      def ant = new AntBuilder()

      def webtest_home = System.properties.'webtest.home'
     
      ant.taskdef(resource:'webtest.taskdef'){
          classpath(){
                 pathelement(location:"$webtest_home/lib")
                 fileset(dir:"$webtest_home/lib", includes:"**/*.jar")
          }
      };     
   
      def config_map = 
['host':"www.somestuff.com",'port':"80",'protocol':"http",'basepath':"",'summary':"true",'saveresponse':"true",'resultfile':"result.xml",'resultpath':".",'haltonfailure':"false",'haltonerror':"false",'defaultPropertyType':"dynamic"]
 
      ant.testSpec(name:gname){
          config(config_map){
             
              option(name:"ThrowExceptionOnScriptError",value:"false")
          }
         
          steps(){
                  invoke(description: "Get the page: somestuff.com", 
url: "http://www.somestuff.com")
            groovy(description:"Store the html of the page","""
                def document = step.context.currentResponse.asXml()
                def props = step.webtestProperties
                props.putAt("htmlPage",document)
                 """)
             invoke(description:"goto W3C 
page",url:"http://validator.w3.org/#validate_by_input")
             setInputField(description:'set textarea with 
feeds',htmlId:'fragment',value="#{htmlPage}")
             clickButton(description:"Submit the simple 
feedvalidation",xpath:"//fieldset[@id='validate-by-input']//input[@value='Check']")
          }

         
      }

What groovy gives to Webtest is an extreme flexibility. You can write 
behaviors that change regards to the type of test you want to make. My 
purpose was to change the type of user registering a given form. I've 
wrote a test suite that filled the value depending on user parameters 
stored in a database, and it was a form that spanned approx. between 50 
and 90  user actions (Ajax site that make appear some fields depending 
on the user profile). And it worked great.
Moreover, I implemented a decision chain that has to decide what value 
to input given a context (example, if the parameter is "Saint Georges 
Sur Baulches" in the database and "St-Georges-Sur-Baulche" on the 
website, you can analyze that in groovy and tell what value to input)

I'll try to answer your questions :
1.) Can Groovy code return data to XML-based tests, somehow? Perhaps via 
ANT properties?
2.) Is it possible to write a class in Groovy, then call its functions 
directly from WebTest XML files?
2.) How does one organize a test suite written in Groovy? It doesn't 
seem that using build.xml etc would be valid.
You have two possibilities :
    1- use the build.xml file with <groovy> step. You can write a class, 
and pass it to the step <groovy file="mygroovyfile.groovy">. Then you 
can exchange data with the ant step via properties :
def props = step.webtestProperties
//get a property from the xml steps
def webtestproperty = props.propertyname
//add a new property that will be visible in the xml test
//be careful it's only strings...
props.put("newpropertyname","newpropertyvalue")

    2- use a groovy class to trigger the step (as I show above). This 
allows you to build complex java objects and services that can be used 
to test the a scenario on the web site

I use both in fact. All my tests are run in Groovy, and I use 
Groovysteps inside to provide deep analysis of the pages and adapt 
behavior regarding of the context

3.) Is it better to create a Groovy test suite using WebTest and the 
WebTest file structure, or is it better to create a Grails project and 
import the WebTest functionality from an external source?
Don't really understand what you mean... Just create your groovy tests 
and run them in a JVM... No need to add Grails upon it unless you used 
this framework to code the website. I do not know much about 
Grails/Webtest integration, so, you can forget what I've just said ;-)

3.) Does anyone have an example/proof-of-concept test suite written 
predominately in Groovy which they might be willing to share?
Can't really share the entire code, because its purpose is really 
precise and might not fit  other needs. But it can be adapt depending on 
the way you want to test. I can give you some tricks if you want

4.) Are there major drawbacks to using a 100% Groovy code base as 
opposed to WebTest's XML based tests?
In fact, i think you gain in flexibility, but you loose a bit of 
abstraction. Groovy is pretty simple and really powerful, but it will 
always be longer to initiate a groovy way of coding than to write XML 
tests...

A last thing : With a 100% Groovy code, you can preprocess a database, 
run your tests against these values, pick results on web pages and then 
compare the tests results immediately with expectations. Groovy is a 
perfect way to analyze quickly XML documents produced by Webtest

Hope that helps

Florent

Pingwy
27, rue des arènes
49100 Angers



Harihara Vinayakaram a écrit :
> Hi
>    I am not qualified to answer all the questions . But let me try 
> answering a few that I know of.  We started with the aim of developing 
> a test suite in groovy but we gave it up since it was difficult to get 
> engineers to learn groovy rather than ant specific syntax which they 
> were familiar. It was easier to get newer tests in
>
>    So we ended up with a test harness in groovy (using the grails 
> framework as the model ) that ran an automated suite by clicking on 
> the links and the rest of the actual tests were in XML .
>
>     Yes XML has its own issues with webtest. Particulary (JIRA WT-382 
> ) which makes verification of tests a pain but gets the job done when 
> integrated with Hudson
>
>    Some issues we faced when we used groovy was :
>        a) It is tough to get the idea that groovy is interpreted . So 
> you see print's in closures getting printed on the console and the 
> actual tests getting executed much later
>
>        b) Organization of tests in groovy did not have any 
> documentation but this was rectified . You can search the mailing list 
> for examples
>
>        c) XPath support in groovy was very laboured . (i.e.) if you 
> want to write a test and then decide on what to click based on some 
> xPath expression was very tough
>
> My experience has been that it is better to stick with XML 
> predominantly and extend if needed using Groovy
>
> Regards
> Hari
>
> On Tue, Jul 1, 2008 at 9:18 PM, <geojohn@lexmark.com 
> <mailto:geojohn@lexmark.com>> wrote:
>
>
>     Hello all,
>
>     I'm researching WebTest's feasibility for use in the testing of
>     some web interfaces. We've achieved some very nice results with
>     WebTest, and we like it, but we also feel that in some cases the
>     use of XML is holding us back. We want functions which accept
>     parameters and return values (includes and properties are not a
>     good substitute), we need more data types (hashes, arrays, etc),
>     we want more extensibility with third-party libraries, etc. It
>     seems that WebTest's integration with Groovy would offer us the
>     desired functionality.
>
>     Unfortunately, as far as I can tell, the use of WebTest with
>     Groovy is not very well-documented. I was hoping that perhaps you
>     could help me out by answering some questions.
>
>     1.) Can Groovy code return data to XML-based tests, somehow?
>     Perhaps via ANT properties?
>     2.) Is it possible to write a class in Groovy, then call its
>     functions directly from WebTest XML files?
>     2.) How does one organize a test suite written in Groovy? It
>     doesn't seem that using build.xml etc would be valid.
>     3.) Is it better to create a Groovy test suite using WebTest and
>     the WebTest file structure, or is it better to create a Grails
>     project and import the WebTest functionality from an external source?
>     3.) Does anyone have an example/proof-of-concept test suite
>     written predominately in Groovy which they might be willing to share?
>     4.) Are there major drawbacks to using a 100% Groovy code base as
>     opposed to WebTest's XML based tests?
>
>     Thank you all for your help. Your input is greatly appreciated.
>
>     Thanks,
>     George Johnson
>
>

--------------000705050207090502050001
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi,<br>
<br>
I take the time to give my point of view, which is not the best one but
some ideas could interest you.<br>
I've built an extensible test suite entirely in Groovy and found it
astonishing.<br>
The based examples are documented, and that's all you need to begin
with. Well, it's just a few lines of code to write in Eclipse <br>
or your favorite IDE&nbsp; (here is an example of how to validate a page
against W3C):<br>
<small><small>&nbsp;&nbsp;&nbsp; &nbsp; def ant = new AntBuilder()<br>
<br>
&nbsp;&nbsp;&nbsp; &nbsp; def webtest_home = System.properties.'webtest.home'<br>
&nbsp;&nbsp;&nbsp; &nbsp; <br>
&nbsp;&nbsp;&nbsp; &nbsp; ant.taskdef(<a class="moz-txt-link-freetext" href="resource:'webtest.taskdef">resource:'webtest.taskdef</a>'){<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; classpath(){<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; pathelement(location:"$webtest_home/lib")<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; fileset(dir:"$webtest_home/lib", includes:"**/*.jar")<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<br>
&nbsp;&nbsp;&nbsp; &nbsp; };&nbsp;&nbsp;&nbsp; &nbsp; <br>
&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; &nbsp; def config_map =
['host':"<a class="moz-txt-link-abbreviated" href="http://www.somestuff.com">www.somestuff.com</a>",'port':"80",'protocol':"http",'basepath':"",'summary':"true",'saveresponse':"true",'resultfile':"result.xml",'resultpath':".",'haltonfailure':"false",'haltonerror':"false",'defaultPropertyType':"dynamic"]<br>
&nbsp;<br>
&nbsp;&nbsp;&nbsp; &nbsp; ant.testSpec(name:gname){<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; config(config_map){<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; <br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; option(name:"ThrowExceptionOnScriptError",value:"false")<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; }<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp; <br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; steps(){<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; invoke(description: "Get the page: somestuff.com",
url: <a class="moz-txt-link-rfc2396E" href="http://www.somestuff.com">"http://www.somestuff.com"</a>)<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; groovy(description:"Store the html of the page","""<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; def document = step.context.currentResponse.asXml()<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; def props = step.webtestProperties<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; props.putAt("htmlPage",document)<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp; """)<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;invoke(description:"goto W3C
page",url:<a class="moz-txt-link-rfc2396E" href="http://validator.w3.org/#validate_by_input">"http://validator.w3.org/#validate_by_input"</a>)<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;setInputField(description:'set textarea with
feeds',htmlId:'fragment',value="#{htmlPage}")<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp; &nbsp;clickButton(description:"Submit the simple
feedvalidation",xpath:"//fieldset[@id='validate-by-input']//input[@value='Check']")<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; }<br>
<br>
&nbsp;&nbsp;&nbsp; &nbsp;&nbsp;&nbsp;&nbsp;&nbsp; <br>
&nbsp;&nbsp;&nbsp; &nbsp; }<br>
<br>
<big><big>What groovy gives to Webtest is an extreme flexibility. You
can write behaviors that change regards to the type of test you want to
make. My purpose was to change the type of user registering a given
form. I've wrote a test suite that filled the value depending on user
parameters stored in a database, and it was a form that spanned approx.
between 50 and 90&nbsp; user actions (Ajax site that make appear some fields
depending on the user profile). And it worked great.<br>
Moreover, I implemented a decision chain that has to decide what value
to input given a context (example, if the parameter is "Saint Georges
Sur Baulches" in the database and "</big></big></small></small><small><small><big><big>St-Georges-Sur-Baulche</big></big></small></small><small><small><big><big>"
on the website, you can analyze that in groovy and tell what value to
input)<br>
<br>
I'll try to answer your questions :<br>
</big></big></small></small><font face="sans-serif" size="2">1.) Can
Groovy code return data to XML-based
tests, somehow? Perhaps via ANT properties?</font>
<br>
<font face="sans-serif" size="2">2.) Is it possible to write a class
in Groovy, then call its functions directly from WebTest XML files?</font>
<br>
<font face="sans-serif" size="2">2.) How does one organize a test suite
written in Groovy? It doesn't seem that using build.xml etc would be
valid.</font><br>
You have two possibilities : <br>
&nbsp;&nbsp;&nbsp; 1- use the build.xml file with &lt;groovy&gt; step. You can write a
class, and pass it to the step &lt;groovy
file="mygroovyfile.groovy"&gt;. Then you can exchange data with the ant
step via properties : <br>
<small><small>def props = step.webtestProperties<br>
//get a property from the xml steps<br>
def webtestproperty = props.propertyname<br>
//add a new property that will be visible in the xml test<br>
//be careful it's only strings...<br>
props.put("newpropertyname","newpropertyvalue")</small></small><br>
<br>
&nbsp;&nbsp;&nbsp; 2- use a groovy class to trigger the step (as I show above). This
allows you to build complex java objects and services that can be used
to test the a scenario on the web site<br>
<br>
I use both in fact. All my tests are run in Groovy, and I use
Groovysteps inside to provide deep analysis of the pages and adapt
behavior regarding of the context<br>
<br>
<font face="sans-serif" size="2">3.) Is it better to create a Groovy
test suite using WebTest and the WebTest file structure, or is it
better
to create a Grails project and import the WebTest functionality from an
external source?</font>
<br>
<big><font face="sans-serif" size="2"><big>Don't really understand what
you mean... Just create your groovy tests and run them in a JVM... No
need to add Grails upon it unless you used this framework to code the
website. I do not know much about Grails/Webtest integration, so, you
can forget what I've just said ;-)</big></font></big><br>
<font face="sans-serif" size="2"><br>
3.) Does anyone have an example/proof-of-concept
test suite written predominately in Groovy which they might be willing
to share?</font>
<br>
Can't really share the entire code, because its purpose is really
precise and might not fit&nbsp; other needs. But it can be adapt depending
on the way you want to test. I can give you some tricks if you want<br>
<br>
<font face="sans-serif" size="2">4.) Are there major drawbacks to using
a 100% Groovy code base as opposed to WebTest's XML based tests?</font>
<br>
In fact, i think you gain in flexibility, but you loose a bit of
abstraction. Groovy is pretty simple and really powerful, but it will
always be longer to initiate a groovy way of coding than to write XML
tests...<br>
<br>
A last thing : With a 100% Groovy code, you can preprocess a database,
run your tests against these values, pick results on web pages and then
compare the tests results immediately with expectations. Groovy is a
perfect way to analyze quickly XML documents produced by Webtest<br>
<br>
Hope that helps<br>
<br>
Florent<br>
<br>
<pre class="moz-signature" cols="72">Pingwy
27, rue des ar&egrave;nes
49100 Angers
</pre>
<br>
<br>
Harihara Vinayakaram a &eacute;crit&nbsp;:
<blockquote
 cite="mid:826eaa920807010941q174abc4ei618eaf0b2584e9ec@mail.gmail.com"
 type="cite">Hi<br>
&nbsp;&nbsp; I am not qualified to answer all the questions . But let me try
answering a few that I know of.&nbsp; We started with the aim of developing
a test suite in groovy but we gave it up since it was difficult to get
engineers to learn groovy rather than ant specific syntax which they
were familiar. It was easier to get newer tests in <br>
  <br>
&nbsp;&nbsp; So we ended up with a test harness in groovy (using the grails
framework as the model ) that ran an automated suite by clicking on the
links and the rest of the actual tests were in XML . <br>
  <br>
&nbsp;&nbsp;&nbsp; Yes XML has its own issues with webtest. Particulary (JIRA WT-382 )
which makes verification of tests a pain but gets the job done when
integrated with Hudson<br>
  <br>
&nbsp;&nbsp; Some issues we faced when we used groovy was :<br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; a) It is tough to get the idea that groovy is interpreted . So
you see print's in closures getting printed on the console and the
actual tests getting executed much later<br>
  <br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; b) Organization of tests in groovy did not have any
documentation but this was rectified . You can search the mailing list
for examples<br>
  <br>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; c) XPath support in groovy was very laboured . (i.e.) if you
want to write a test and then decide on what to click based on some
xPath expression was very tough<br>
  <br>
My experience has been that it is better to stick with XML
predominantly and extend if needed using Groovy<br>
  <br>
Regards<br>
Hari<br>
  <br>
  <div class="gmail_quote">On Tue, Jul 1, 2008 at 9:18 PM, &lt;<a
 moz-do-not-send="true" href="mailto:geojohn@lexmark.com">geojohn@lexmark.com</a>&gt;
wrote:<br>
  <blockquote class="gmail_quote"
 style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
    <br>
    <font face="sans-serif" size="2">Hello all,</font>
    <br>
    <br>
    <font face="sans-serif" size="2">I'm researching WebTest's
feasibility
for use in the testing of some web interfaces. We've achieved some very
nice results with WebTest, and we like it, but we also feel that in
some
cases the use of XML is holding us back. We want functions which accept
parameters and return values (includes and properties are not a good
substitute),
we need more data types (hashes, arrays, etc), we want more
extensibility
with third-party libraries, etc. It seems that WebTest's integration
with
Groovy would offer us the desired functionality.</font>
    <br>
    <br>
    <font face="sans-serif" size="2">Unfortunately, as far as I can
tell,
the use of WebTest with Groovy is not very well-documented. I was
hoping
that perhaps you could help me out by answering some questions.</font>
    <br>
    <br>
    <font face="sans-serif" size="2">1.) Can Groovy code return data to
XML-based
tests, somehow? Perhaps via ANT properties?</font>
    <br>
    <font face="sans-serif" size="2">2.) Is it possible to write a
class
in Groovy, then call its functions directly from WebTest XML files?</font>
    <br>
    <font face="sans-serif" size="2">2.) How does one organize a test
suite
written in Groovy? It doesn't seem that using build.xml etc would be
valid.</font>
    <br>
    <font face="sans-serif" size="2">3.) Is it better to create a
Groovy
test suite using WebTest and the WebTest file structure, or is it
better
to create a Grails project and import the WebTest functionality from an
external source?</font>
    <br>
    <font face="sans-serif" size="2">3.) Does anyone have an
example/proof-of-concept
test suite written predominately in Groovy which they might be willing
to share?</font>
    <br>
    <font face="sans-serif" size="2">4.) Are there major drawbacks to
using
a 100% Groovy code base as opposed to WebTest's XML based tests?</font>
    <br>
    <br>
    <font face="sans-serif" size="2">Thank you all for your help. Your
input
is greatly appreciated.</font>
    <br>
    <br>
    <font face="sans-serif" size="2">Thanks,</font>
    <br>
    <font face="sans-serif" size="2">George Johnson</font></blockquote>
  </div>
  <br>
</blockquote>
</body>
</html>

--------------000705050207090502050001--