[Webtest] How to check for broken images?
Marc Guillemot
webtest@lists.canoo.com
Sat, 24 May 2003 14:36:37 +0200
This is a multi-part message in MIME format.
------=_NextPart_000_013A_01C32201.E2073260
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Oups, sorry, attached are the 2 files I've added/modified to have a
<verifyimages> step.
- in com.canoo.webtest.ant.TestStepSequence added addVerifyimages(..)
- created com.canoo.webtest.steps.verify.VerifyImages (inspired from
VerifyLinks, not cleaned and therefore absolute not ready to be integrated)
I hope this helps,
Marc.
----- Original Message -----
From: <cowden@charter.net>
To: <webtest@lists.canoo.com>
Sent: Saturday, May 24, 2003 2:06 PM
Subject: RE: [Webtest] How to check for broken images?
> Marc:
>
> I'd love to get that source! Please mail it to me.
> Thanks
> Dave
>
> -----Original Message-----
> From: webtest-admin@lists.canoo.com
> [mailto:webtest-admin@lists.canoo.com]On Behalf Of Marc Guillemot
> Sent: Thursday, May 22, 2003 2:29 AM
> To: webtest@lists.canoo.com
> Subject: Re: [Webtest] How to check for broken images?
>
>
> Hi Dave,
>
> I've faced the same problem and I customized my local copy of Webtest to
add
> a <verifyimages> test step which I can mail you, if you want.
>
> Marc.
>
> ----- Original Message -----
> From: <cowden@charter.net>
> To: <webtest@lists.canoo.com>
> Sent: Thursday, May 22, 2003 6:19 AM
> Subject: [Webtest] How to check for broken images?
>
>
> > Hello:
> >
> > I'm pretty new to webtest. After reading the docs, the list archives,
and
> > the javadoc, I'm unable to figure out how to go about verifying that all
> of
> > the images on a page are available. <verifylinks> didnt seem to catch
> > broken images.
> >
> > Ideally, I'd like to be able to download and verify every image on the
> page,
> > to make sure that they are all available.
> >
> > Has anyone done this?
> >
> > Thanks
> > Dave
> >
> > _______________________________________________
> > WebTest mailing list
> > WebTest@lists.canoo.com
> > http://lists.canoo.com/mailman/listinfo/webtest
> >
>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>
------=_NextPart_000_013A_01C32201.E2073260
Content-Type: application/octet-stream;
name="TestStepSequence.java"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="TestStepSequence.java"
// Copyright =A9 2002 Canoo Engineering AG, Switzerland.=0A=
package com.canoo.webtest.ant;=0A=
=0A=
import com.canoo.webtest.interfaces.IStepSequence;=0A=
import com.canoo.webtest.steps.*;=0A=
import com.canoo.webtest.steps.verify.*;=0A=
import com.canoo.webtest.steps.request.*;=0A=
=0A=
import java.io.*;=0A=
import java.util.*;=0A=
=0A=
/**=0A=
* This is a container class for a list of =
AbstractTestSpecificationSteps. It is used for=0A=
* the <code><steps></code> element in ant build scripts as well =
as for the=0A=
* <code><not></code> test step elements.=0A=
*=0A=
* <p>An instance of this class is usually created by ant if it =
encounters the nested=0A=
* <code><steps></code> element within a =
<code><testSpec></code> element.=0A=
* For each nested element within the <code><steps></code> element =
is the=0A=
* appropriate <code>addXxx</code> method called. Ant matches the nested =
element name to the=0A=
* to the method name (e.g. <code><clickbutton></code> is mapped to=0A=
* <code>addClickbutton</code> and instantiates an object according to =
the signature of=0A=
* this method, e.g. a new instance of ClickButton.</p>=0A=
*=0A=
* <p>Since there is not 1:1 association between test step element names =
and test step=0A=
* implementation classes it is necessary to set the step type in order =
to allow=0A=
* later a mapping back to the original test step for meaningful =
reporting.</p>=0A=
*=0A=
* <p><em>Note: The nested element object <code>ClickButton</code> is =
instantiated and "added" to the=0A=
* <code>TestStepSequence</code> before the nested element is completely =
parsed! No attributes=0A=
* or nested elements of the <code><clickbutton></code> elements =
are available at time=0A=
* when it is added to <code>TestStepSequence</code>.</em></p>=0A=
*=0A=
* @see com.canoo.webtest.ant.TestSpecificationTask=0A=
*=0A=
* @author Carsten Seibert=0A=
*/=0A=
public class TestStepSequence extends Object implements IStepSequence, =
Serializable, Cloneable {=0A=
private List fSteps;=0A=
private String fStepId;=0A=
=0A=
public TestStepSequence() {=0A=
setSteps(new ArrayList());=0A=
}=0A=
=0A=
public void addFollowframe(FollowFrame step) {=0A=
step.setStepType("followframe");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addClickbutton(ClickButton step) {=0A=
step.setStepType("clickbutton");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addClicklink(ClickLink step) {=0A=
step.setStepType("clicklink");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifylinks(VerifyLinks step) {=0A=
step.setStepType("verifylinks");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifyimages(VerifyImages step) {=0A=
step.setStepType("verifyimages");=0A=
addStep(step);=0A=
}=0A=
=0A=
=0A=
public void addVerifyxpath(VerifyXPath step) {=0A=
step.setStepType("verifyxpath");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifycheckbox(VerifyCheckbox step) {=0A=
step.setStepType("verifycheckbox");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addInvoke(InvokePage step) {=0A=
step.setStepType("invoke");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addSetinputfield(SetInputField step) {=0A=
step.setStepType("setinputfield");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addSetselectfield(SetSelectField step) {=0A=
step.setStepType("setselectfield");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addSetcheckbox(SetCheckbox step) {=0A=
step.setStepType("setcheckbox");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addStep(Step step) {=0A=
fSteps.add(step);=0A=
}=0A=
=0A=
public void addVerifyelement(VerifyHtmlElementNamed step) {=0A=
step.setStepType("verifyelement");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifyelementtext(VerifyElementText step) {=0A=
step.setStepType("verifyelementtext");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifyinputfield(VerifyInputField step) {=0A=
step.setStepType("verifyinputfield");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifyselectfield(VerifySelectField step) {=0A=
step.setStepType("verifyselectfield");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifytext(VerifyText step) {=0A=
step.setStepType("verifytext");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifytextarea(VerifyElementText step) {=0A=
step.setStepType("verifytextarea");=0A=
step.setType(Step.ELEMENT_TYPE_TEXTAREA);=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addVerifytitle(VerifyElementText step) {=0A=
step.setStepType("verifytitle");=0A=
step.setType(Step.ELEMENT_TYPE_TITLE);=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addStoreregex(StoreRegExMatch step) {=0A=
step.setStepType("storeregex");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addStorexpath(StoreXPath step) {=0A=
step.setStepType("storexpath");=0A=
addStep(step);=0A=
}=0A=
=0A=
public String getStepId() {=0A=
return fStepId;=0A=
}=0A=
=0A=
public List getSteps() {=0A=
return fSteps;=0A=
}=0A=
=0A=
public void setStepId(String stepId) {=0A=
fStepId =3D stepId;=0A=
}=0A=
=0A=
public void setSteps(List newSteps) {=0A=
fSteps =3D newSteps;=0A=
}=0A=
=0A=
/**=0A=
* A <code><not></code> element is a container for an =
arbitrary list of=0A=
* other test steps, i.e. they can contain all steps that a =
<code><steps></code>=0A=
* element can have. Therefore is a <code><not></code> element =
"parsed" as=0A=
* <code>TestStepSequence</code> (like <code><steps></code>) =
and this method=0A=
* creates the correct test step instance <code>FailWrapper</code>.=0A=
*=0A=
* @param failSteps A sequence of steps that must fail=0A=
*=0A=
* @see com.canoo.webtest.steps.FailWrapper=0A=
*/=0A=
public void addNot(TestStepSequence failSteps) {=0A=
FailWrapper step =3D new FailWrapper(failSteps);=0A=
step.setStepType("not");=0A=
addStep(step);=0A=
}=0A=
=0A=
/**=0A=
*=0A=
*=0A=
* @see com.canoo.webtest.steps.RepeatWrapper=0A=
*/=0A=
public void addRepeat(RepeatStep repeatStep) {=0A=
RepeatWrapper step =3D new RepeatWrapper(repeatStep);=0A=
step.setStepType("repeat");=0A=
addStep(step);=0A=
}=0A=
=0A=
public void addPreviousresponse(PreviousResponse step) {=0A=
step.setStepType("previousresponse");=0A=
addStep(step);=0A=
}=0A=
=0A=
public String toString() {=0A=
StringBuffer sb =3D new StringBuffer();=0A=
sb.append(getClass().getName());=0A=
sb.append("(stepId=3D\"");=0A=
sb.append(getStepId());=0A=
sb.append("\", wrappedSteps=3D[");=0A=
for (Iterator iterator =3D fSteps.iterator(); =
iterator.hasNext();) {=0A=
sb.append("<");=0A=
sb.append(iterator.next().toString());=0A=
sb.append("> ");=0A=
}=0A=
sb.append("])");=0A=
return sb.toString();=0A=
}=0A=
=0A=
/**=0A=
* Clone the step list by deepcopying it.=0A=
*/=0A=
public Object clone() {=0A=
TestStepSequence newClone;=0A=
=0A=
try {=0A=
newClone =3D (TestStepSequence) super.clone();=0A=
newClone.fSteps =3D (List) deepCopyObject(fSteps);=0A=
} catch (Exception e) {=0A=
throw new IllegalStateException(e.toString());=0A=
}=0A=
=0A=
=0A=
return newClone;=0A=
}=0A=
=0A=
/**=0A=
* Use serialization to create a deepcopy of a given object, thus=0A=
* it must implement the serializable interface.=0A=
*=0A=
* @param object A serializable object.=0A=
*/=0A=
private Object deepCopyObject(Object object) throws Exception {=0A=
Object result;=0A=
=0A=
ByteArrayOutputStream baos =3D new ByteArrayOutputStream();=0A=
ObjectOutputStream oos =3D new ObjectOutputStream(baos);=0A=
oos.writeObject(object);=0A=
ByteArrayInputStream bais =3D new =
ByteArrayInputStream(baos.toByteArray());=0A=
ObjectInputStream ois =3D new ObjectInputStream(bais);=0A=
result =3D ois.readObject();=0A=
=0A=
return result;=0A=
}=0A=
}=0A=
------=_NextPart_000_013A_01C32201.E2073260
Content-Type: application/octet-stream;
name="VerifyImages.java"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="VerifyImages.java"
package com.canoo.webtest.steps.verify;=0A=
=0A=
import java.io.IOException;=0A=
import java.net.MalformedURLException;=0A=
import java.util.ArrayList;=0A=
import java.util.Iterator;=0A=
import java.util.LinkedList;=0A=
import java.util.List;=0A=
import java.util.Vector;=0A=
=0A=
import org.xml.sax.SAXException;=0A=
=0A=
import com.canoo.webtest.engine.Context;=0A=
import com.canoo.webtest.engine.StepExecutionException;=0A=
import com.canoo.webtest.engine.StepFailedException;=0A=
import com.canoo.webtest.steps.Step;=0A=
import com.meterware.httpunit.HttpException;=0A=
import com.meterware.httpunit.WebConversation;=0A=
import com.meterware.httpunit.WebImage;=0A=
import com.meterware.httpunit.WebLink;=0A=
import com.meterware.httpunit.WebRequest;=0A=
import com.meterware.httpunit.WebResponse;=0A=
=0A=
/**=0A=
@author Marc Guillemot=0A=
**/=0A=
=0A=
public class VerifyImages extends Step=0A=
{=0A=
WebResponse fLastResponse;=0A=
StringBuffer fBrokenURLs =3D new StringBuffer(200);=0A=
int fMaxDepth =3D 0;=0A=
int fCurrentDepth =3D 0;=0A=
boolean fOnSiteOnly =3D false;=0A=
Context fContext =3D null;=0A=
=0A=
public void doExecute(Context context)=0A=
{=0A=
fContext =3D context;=0A=
WebConversation conversation =3D context.getWebConversation();=0A=
fLastResponse =3D context.getLastResponse();=0A=
=0A=
List liFailedImagesUrl =3D checkImages(conversation, fLastResponse);=0A=
if (!liFailedImagesUrl.isEmpty())=0A=
{=0A=
StringBuffer sb =3D=0A=
new StringBuffer(=0A=
getStepId(context)=0A=
+ ": missing images for "=0A=
+ fLastResponse.getURL()=0A=
+ ":");=0A=
for (Iterator iter =3D liFailedImagesUrl.iterator(); iter.hasNext();)=0A=
{=0A=
sb.append(iter.next()).append(";\r\n");=0A=
}=0A=
throw new StepFailedException(sb.toString(), this);=0A=
}=0A=
=0A=
try=0A=
{=0A=
WebImage[] tabImg =3D fLastResponse.getImages();=0A=
System.out.println(=0A=
"tabImg: " + tabImg + " -> " + tabImg.length + " images");=0A=
for (int i =3D 0; i < tabImg.length; ++i)=0A=
{=0A=
System.out.println(tabImg[i].getRequest().getURL());=0A=
WebResponse resp =3D=0A=
conversation.sendRequest(tabImg[i].getRequest());=0A=
}=0A=
if (true)=0A=
throw new StepFailedException(=0A=
getStepId(context) + ": missing images: " + tabImg,=0A=
this);=0A=
StringBuffer sb =3D new StringBuffer();=0A=
}=0A=
catch (Exception e)=0A=
{=0A=
throw new StepExecutionException(=0A=
"Exception occured: " + e.toString());=0A=
}=0A=
=0A=
checkVisits(conversation, fLastResponse);=0A=
if (fBrokenURLs.length() > 0)=0A=
{=0A=
throw new StepFailedException(=0A=
getStepId(context)=0A=
+ ": broken Link(s) : "=0A=
+ fBrokenURLs.toString(),=0A=
this);=0A=
}=0A=
}=0A=
=0A=
/**=0A=
* Checks the images and return the url of the missing ones=0A=
*/=0A=
protected List checkImages(=0A=
WebConversation _conversation,=0A=
WebResponse _response)=0A=
{=0A=
List liRep =3D new ArrayList();=0A=
WebImage[] tabImg =3D null;=0A=
try=0A=
{=0A=
tabImg =3D _response.getImages();=0A=
}=0A=
catch (SAXException e)=0A=
{=0A=
throw new StepExecutionException(=0A=
"Exception occured: " + e.toString());=0A=
}=0A=
=0A=
for (int i =3D 0; i < tabImg.length; ++i)=0A=
{=0A=
try=0A=
{=0A=
WebResponse resp =3D=0A=
_conversation.sendRequest(tabImg[i].getRequest());=0A=
}=0A=
catch (Exception e)=0A=
{=0A=
liRep.add(tabImg[i].getSource());=0A=
}=0A=
}=0A=
=0A=
return liRep;=0A=
}=0A=
=0A=
public void checkVisits(WebConversation conversation, WebResponse =
response)=0A=
{=0A=
Visit[] visits =3D new Visit[0];=0A=
try=0A=
{=0A=
visits =3D getVisits(conversation, response);=0A=
storeBrokenUrls(visits, response);=0A=
}=0A=
catch (Exception e)=0A=
{=0A=
handleUnexpectedException(e);=0A=
}=0A=
}=0A=
=0A=
protected void storeBrokenUrls(Visit[] visits, WebResponse response)=0A=
throws MalformedURLException=0A=
{=0A=
for (int i =3D 0; i < visits.length; i++)=0A=
{=0A=
if (visits[i].isNotBroken() =3D=3D false)=0A=
{=0A=
fBrokenURLs.append(visits[i].getRequest().getURL().toString());=0A=
fBrokenURLs.append(" on ");=0A=
fBrokenURLs.append(response.getURL().toString());=0A=
fBrokenURLs.append("; ");=0A=
}=0A=
}=0A=
}=0A=
=0A=
public int getLinkCount(WebResponse response)=0A=
throws org.xml.sax.SAXException=0A=
{=0A=
return getGoodLinks(response).length;=0A=
}=0A=
=0A=
private WebLink[] getGoodLinks(WebResponse response)=0A=
throws org.xml.sax.SAXException=0A=
{=0A=
WebLink[] links =3D response.getLinks();=0A=
Vector linkVector =3D new Vector();=0A=
=0A=
for (int i =3D 0; i < links.length; i++)=0A=
{=0A=
try=0A=
{=0A=
if (links[i]=0A=
.getRequest()=0A=
.getURL()=0A=
.getProtocol()=0A=
.equals("http"))=0A=
{=0A=
linkVector.addElement(links[i]);=0A=
}=0A=
}=0A=
catch (MalformedURLException e)=0A=
{=0A=
}=0A=
}=0A=
=0A=
WebLink[] goodLinks =3D new WebLink[linkVector.size()];=0A=
for (int j =3D 0; j < linkVector.size(); j++)=0A=
{=0A=
goodLinks[j] =3D (WebLink) linkVector.elementAt(j);=0A=
}=0A=
return goodLinks;=0A=
}=0A=
=0A=
public WebRequest[] getRequests(WebResponse response) throws =
SAXException=0A=
{=0A=
WebLink[] links =3D getGoodLinks(response);=0A=
WebRequest[] requests =3D new WebRequest[links.length];=0A=
for (int i =3D 0; i < requests.length; i++)=0A=
{=0A=
requests[i] =3D links[i].getRequest();=0A=
}=0A=
return requests;=0A=
}=0A=
=0A=
public Visit[] getVisits(=0A=
WebConversation conversation,=0A=
WebResponse response)=0A=
throws MalformedURLException, SAXException=0A=
{=0A=
WebRequest[] requests =3D getRequests(response);=0A=
List collector =3D new LinkedList();=0A=
for (int i =3D 0; i < requests.length; i++)=0A=
{=0A=
WebRequest request =3D requests[i];=0A=
if (!visited(request, collector))=0A=
{=0A=
collector.add(visit(request, conversation));=0A=
}=0A=
}=0A=
return (Visit[]) collector.toArray(new Visit[collector.size()]);=0A=
}=0A=
=0A=
//todo: better use a Set and have equals() and hashcode() in Visit go =
to getRequest().getURL().toExternalForm()=0A=
private boolean visited(WebRequest request, List visits)=0A=
throws MalformedURLException=0A=
{=0A=
for (Iterator iterator =3D visits.iterator(); iterator.hasNext();)=0A=
{=0A=
Visit visit =3D (Visit) iterator.next();=0A=
if (request=0A=
.getURL()=0A=
.toExternalForm()=0A=
.equals(visit.getRequest().getURL().toExternalForm()))=0A=
{=0A=
return true;=0A=
}=0A=
}=0A=
return false;=0A=
}=0A=
=0A=
Visit visit(WebRequest request, WebConversation conversation)=0A=
throws MalformedURLException, SAXException=0A=
{=0A=
Visit visitResult =3D new Visit(request);=0A=
visitResult.setNotBroken(false);=0A=
try=0A=
{=0A=
logText(fContext, request.getURL().toExternalForm());=0A=
WebResponse response =3D conversation.getResponse(request);=0A=
visitResult.setResponse(response);=0A=
visitResult.setNotBroken(response.getResponseCode() < 400);=0A=
}=0A=
catch (SAXException e)=0A=
{=0A=
visitResult.setNotBroken(true);=0A=
}=0A=
catch (HttpException e)=0A=
{=0A=
visitResult.setNotBroken(false);=0A=
}=0A=
catch (IOException e)=0A=
{=0A=
visitResult.setNotBroken(false);=0A=
}=0A=
=0A=
followRecursively(visitResult, conversation);=0A=
=0A=
return visitResult;=0A=
}=0A=
=0A=
protected void followRecursively(=0A=
Visit visitResult,=0A=
WebConversation conversation)=0A=
throws MalformedURLException, SAXException=0A=
{=0A=
if (visitResult.isNotBroken() && fCurrentDepth < fMaxDepth)=0A=
{=0A=
if (stopHunting(visitResult) =3D=3D false)=0A=
{=0A=
fCurrentDepth++;=0A=
checkVisits(conversation, visitResult.getResponse());=0A=
fCurrentDepth--;=0A=
}=0A=
}=0A=
}=0A=
=0A=
protected boolean stopHunting(Visit visitResult)=0A=
{=0A=
if (fOnSiteOnly && foreignHost(visitResult))=0A=
return true;=0A=
if (noWebPage(visitResult))=0A=
return true;=0A=
return false;=0A=
}=0A=
=0A=
protected boolean noWebPage(Visit visitResult)=0A=
{=0A=
return visitResult.getResponse().getContentType().indexOf("html") < 0;=0A=
}=0A=
=0A=
protected boolean foreignHost(Visit visitResult)=0A=
{=0A=
return visitResult.getResponse().getURL().getHost().equals(=0A=
getBaseResponse().getURL().getHost())=0A=
=3D=3D false;=0A=
}=0A=
=0A=
public WebResponse getBaseResponse()=0A=
{=0A=
return fLastResponse;=0A=
}=0A=
=0A=
//todo: go for int ?=0A=
public void setDepth(String depth)=0A=
{=0A=
//dk: check for validity=0A=
fMaxDepth =3D Integer.parseInt(depth);=0A=
}=0A=
=0A=
public void setOnSiteOnly(String onSiteOnly)=0A=
{=0A=
// todo: maybe call generic boolean handling=0A=
if (onSiteOnly.equals("true"))=0A=
fOnSiteOnly =3D true;=0A=
}=0A=
=0A=
}
------=_NextPart_000_013A_01C32201.E2073260--