[Webtest] How to check for broken images?

Marc Guillemot webtest@lists.canoo.com
Sat, 24 May 2003 14:36:37 +0200


This is a multi-part message in MIME format.

------=_NextPart_000_013A_01C32201.E2073260
Content-Type: text/plain;
	charset="iso-8859-1"
Content-Transfer-Encoding: 7bit

Oups, sorry, attached are the 2 files I've added/modified to have a
<verifyimages> step.

- in com.canoo.webtest.ant.TestStepSequence added addVerifyimages(..)
- created com.canoo.webtest.steps.verify.VerifyImages (inspired from
VerifyLinks, not cleaned and therefore absolute not ready to be integrated)

I hope this helps,

Marc.


----- Original Message -----
From: <cowden@charter.net>
To: <webtest@lists.canoo.com>
Sent: Saturday, May 24, 2003 2:06 PM
Subject: RE: [Webtest] How to check for broken images?


> Marc:
>
> I'd love to get that source! Please mail it to me.
> Thanks
> Dave
>
> -----Original Message-----
> From: webtest-admin@lists.canoo.com
> [mailto:webtest-admin@lists.canoo.com]On Behalf Of Marc Guillemot
> Sent: Thursday, May 22, 2003 2:29 AM
> To: webtest@lists.canoo.com
> Subject: Re: [Webtest] How to check for broken images?
>
>
> Hi Dave,
>
> I've faced the same problem and I customized my local copy of Webtest to
add
> a <verifyimages> test step which I can mail you, if you want.
>
> Marc.
>
> ----- Original Message -----
> From: <cowden@charter.net>
> To: <webtest@lists.canoo.com>
> Sent: Thursday, May 22, 2003 6:19 AM
> Subject: [Webtest] How to check for broken images?
>
>
> > Hello:
> >
> > I'm pretty new to webtest. After reading the docs, the list archives,
and
> > the javadoc, I'm unable to figure out how to go about verifying that all
> of
> > the images on a page are available.  <verifylinks> didnt seem to catch
> > broken images.
> >
> > Ideally, I'd like to be able to download and verify every image on the
> page,
> > to make sure that they are all available.
> >
> > Has anyone done this?
> >
> > Thanks
> > Dave
> >
> > _______________________________________________
> > WebTest mailing list
> > WebTest@lists.canoo.com
> > http://lists.canoo.com/mailman/listinfo/webtest
> >
>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>
> _______________________________________________
> WebTest mailing list
> WebTest@lists.canoo.com
> http://lists.canoo.com/mailman/listinfo/webtest
>

------=_NextPart_000_013A_01C32201.E2073260
Content-Type: application/octet-stream;
	name="TestStepSequence.java"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="TestStepSequence.java"

// Copyright =A9 2002 Canoo Engineering AG, Switzerland.=0A=
package com.canoo.webtest.ant;=0A=
=0A=
import com.canoo.webtest.interfaces.IStepSequence;=0A=
import com.canoo.webtest.steps.*;=0A=
import com.canoo.webtest.steps.verify.*;=0A=
import com.canoo.webtest.steps.request.*;=0A=
=0A=
import java.io.*;=0A=
import java.util.*;=0A=
=0A=
/**=0A=
 * This is a container class for a list of =
AbstractTestSpecificationSteps. It is used for=0A=
 * the <code>&lt;steps&gt;</code> element in ant build scripts as well =
as for the=0A=
 * <code>&lt;not&gt;</code> test step elements.=0A=
 *=0A=
 * <p>An instance of this class is usually created by ant if it =
encounters the nested=0A=
 * <code>&lt;steps&gt;</code> element within a =
<code>&lt;testSpec&gt;</code> element.=0A=
 * For each nested element within the <code>&lt;steps&gt;</code> element =
is the=0A=
 * appropriate <code>addXxx</code> method called. Ant matches the nested =
element name to the=0A=
 * to the method name (e.g. <code>&lt;clickbutton&gt;</code> is mapped to=0A=
 * <code>addClickbutton</code> and instantiates an object according to =
the signature of=0A=
 * this method, e.g. a new instance of ClickButton.</p>=0A=
 *=0A=
 * <p>Since there is not 1:1 association between test step element names =
and test step=0A=
 * implementation classes it is necessary to set the step type in order =
to allow=0A=
 * later a mapping back to the original test step for meaningful =
reporting.</p>=0A=
 *=0A=
 * <p><em>Note: The nested element object <code>ClickButton</code> is =
instantiated and "added" to the=0A=
 * <code>TestStepSequence</code> before the nested element is completely =
parsed! No attributes=0A=
 * or nested elements of the <code>&lt;clickbutton&gt;</code> elements =
are available at time=0A=
 * when it is added to <code>TestStepSequence</code>.</em></p>=0A=
 *=0A=
 * @see com.canoo.webtest.ant.TestSpecificationTask=0A=
 *=0A=
 * @author Carsten Seibert=0A=
 */=0A=
public class TestStepSequence extends Object implements IStepSequence, =
Serializable, Cloneable {=0A=
    private List fSteps;=0A=
    private String fStepId;=0A=
=0A=
    public TestStepSequence() {=0A=
        setSteps(new ArrayList());=0A=
    }=0A=
=0A=
    public void addFollowframe(FollowFrame step) {=0A=
        step.setStepType("followframe");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addClickbutton(ClickButton step) {=0A=
        step.setStepType("clickbutton");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addClicklink(ClickLink step) {=0A=
        step.setStepType("clicklink");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifylinks(VerifyLinks step) {=0A=
        step.setStepType("verifylinks");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifyimages(VerifyImages step) {=0A=
        step.setStepType("verifyimages");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
=0A=
    public void addVerifyxpath(VerifyXPath step) {=0A=
        step.setStepType("verifyxpath");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifycheckbox(VerifyCheckbox step) {=0A=
        step.setStepType("verifycheckbox");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addInvoke(InvokePage step) {=0A=
        step.setStepType("invoke");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addSetinputfield(SetInputField step) {=0A=
        step.setStepType("setinputfield");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addSetselectfield(SetSelectField step) {=0A=
        step.setStepType("setselectfield");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addSetcheckbox(SetCheckbox step) {=0A=
        step.setStepType("setcheckbox");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addStep(Step step) {=0A=
        fSteps.add(step);=0A=
    }=0A=
=0A=
    public void addVerifyelement(VerifyHtmlElementNamed step) {=0A=
        step.setStepType("verifyelement");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifyelementtext(VerifyElementText step) {=0A=
        step.setStepType("verifyelementtext");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifyinputfield(VerifyInputField step) {=0A=
        step.setStepType("verifyinputfield");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifyselectfield(VerifySelectField step) {=0A=
        step.setStepType("verifyselectfield");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifytext(VerifyText step) {=0A=
        step.setStepType("verifytext");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifytextarea(VerifyElementText step) {=0A=
        step.setStepType("verifytextarea");=0A=
        step.setType(Step.ELEMENT_TYPE_TEXTAREA);=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addVerifytitle(VerifyElementText step) {=0A=
        step.setStepType("verifytitle");=0A=
        step.setType(Step.ELEMENT_TYPE_TITLE);=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addStoreregex(StoreRegExMatch step) {=0A=
        step.setStepType("storeregex");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addStorexpath(StoreXPath step) {=0A=
        step.setStepType("storexpath");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public String getStepId() {=0A=
        return fStepId;=0A=
    }=0A=
=0A=
    public List getSteps() {=0A=
        return fSteps;=0A=
    }=0A=
=0A=
    public void setStepId(String stepId) {=0A=
        fStepId =3D stepId;=0A=
    }=0A=
=0A=
    public void setSteps(List newSteps) {=0A=
        fSteps =3D newSteps;=0A=
    }=0A=
=0A=
    /**=0A=
     * A <code>&lt;not&gt;</code> element is a container for an =
arbitrary list of=0A=
     * other test steps, i.e. they can contain all steps that a =
<code>&lt;steps&gt;</code>=0A=
     * element can have. Therefore is a <code>&lt;not&gt;</code> element =
"parsed" as=0A=
     * <code>TestStepSequence</code> (like <code>&lt;steps&gt;</code>) =
and this method=0A=
     * creates the correct test step instance <code>FailWrapper</code>.=0A=
     *=0A=
     * @param failSteps A sequence of steps that must fail=0A=
     *=0A=
     * @see com.canoo.webtest.steps.FailWrapper=0A=
     */=0A=
    public void addNot(TestStepSequence failSteps) {=0A=
        FailWrapper step =3D new FailWrapper(failSteps);=0A=
        step.setStepType("not");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    /**=0A=
     *=0A=
     *=0A=
     * @see com.canoo.webtest.steps.RepeatWrapper=0A=
     */=0A=
    public void addRepeat(RepeatStep repeatStep) {=0A=
        RepeatWrapper step =3D new RepeatWrapper(repeatStep);=0A=
        step.setStepType("repeat");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public void addPreviousresponse(PreviousResponse step) {=0A=
        step.setStepType("previousresponse");=0A=
        addStep(step);=0A=
    }=0A=
=0A=
    public String toString() {=0A=
        StringBuffer sb =3D new StringBuffer();=0A=
        sb.append(getClass().getName());=0A=
        sb.append("(stepId=3D\"");=0A=
        sb.append(getStepId());=0A=
        sb.append("\", wrappedSteps=3D[");=0A=
        for (Iterator iterator =3D fSteps.iterator(); =
iterator.hasNext();) {=0A=
            sb.append("<");=0A=
            sb.append(iterator.next().toString());=0A=
            sb.append("> ");=0A=
        }=0A=
        sb.append("])");=0A=
        return sb.toString();=0A=
    }=0A=
=0A=
    /**=0A=
     * Clone the step list by deepcopying it.=0A=
     */=0A=
    public Object clone() {=0A=
        TestStepSequence newClone;=0A=
=0A=
        try {=0A=
            newClone =3D (TestStepSequence) super.clone();=0A=
            newClone.fSteps =3D (List) deepCopyObject(fSteps);=0A=
        } catch (Exception e) {=0A=
            throw new IllegalStateException(e.toString());=0A=
        }=0A=
=0A=
=0A=
        return newClone;=0A=
    }=0A=
=0A=
    /**=0A=
     * Use serialization to create a deepcopy of a given object, thus=0A=
     * it must implement the serializable interface.=0A=
     *=0A=
     * @param object A serializable object.=0A=
     */=0A=
    private Object deepCopyObject(Object object) throws Exception {=0A=
        Object result;=0A=
=0A=
        ByteArrayOutputStream baos =3D new ByteArrayOutputStream();=0A=
        ObjectOutputStream oos =3D new ObjectOutputStream(baos);=0A=
        oos.writeObject(object);=0A=
        ByteArrayInputStream bais =3D new =
ByteArrayInputStream(baos.toByteArray());=0A=
        ObjectInputStream ois =3D new ObjectInputStream(bais);=0A=
        result =3D ois.readObject();=0A=
=0A=
        return result;=0A=
    }=0A=
}=0A=

------=_NextPart_000_013A_01C32201.E2073260
Content-Type: application/octet-stream;
	name="VerifyImages.java"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
	filename="VerifyImages.java"

package com.canoo.webtest.steps.verify;=0A=
=0A=
import java.io.IOException;=0A=
import java.net.MalformedURLException;=0A=
import java.util.ArrayList;=0A=
import java.util.Iterator;=0A=
import java.util.LinkedList;=0A=
import java.util.List;=0A=
import java.util.Vector;=0A=
=0A=
import org.xml.sax.SAXException;=0A=
=0A=
import com.canoo.webtest.engine.Context;=0A=
import com.canoo.webtest.engine.StepExecutionException;=0A=
import com.canoo.webtest.engine.StepFailedException;=0A=
import com.canoo.webtest.steps.Step;=0A=
import com.meterware.httpunit.HttpException;=0A=
import com.meterware.httpunit.WebConversation;=0A=
import com.meterware.httpunit.WebImage;=0A=
import com.meterware.httpunit.WebLink;=0A=
import com.meterware.httpunit.WebRequest;=0A=
import com.meterware.httpunit.WebResponse;=0A=
=0A=
/**=0A=
 @author Marc Guillemot=0A=
**/=0A=
=0A=
public class VerifyImages extends Step=0A=
{=0A=
	WebResponse fLastResponse;=0A=
	StringBuffer fBrokenURLs =3D new StringBuffer(200);=0A=
	int fMaxDepth =3D 0;=0A=
	int fCurrentDepth =3D 0;=0A=
	boolean fOnSiteOnly =3D false;=0A=
	Context fContext =3D null;=0A=
=0A=
	public void doExecute(Context context)=0A=
	{=0A=
		fContext =3D context;=0A=
		WebConversation conversation =3D context.getWebConversation();=0A=
		fLastResponse =3D context.getLastResponse();=0A=
=0A=
		List liFailedImagesUrl =3D checkImages(conversation, fLastResponse);=0A=
		if (!liFailedImagesUrl.isEmpty())=0A=
		{=0A=
			StringBuffer sb =3D=0A=
				new StringBuffer(=0A=
					getStepId(context)=0A=
						+ ": missing images for "=0A=
						+ fLastResponse.getURL()=0A=
						+ ":");=0A=
			for (Iterator iter =3D liFailedImagesUrl.iterator(); iter.hasNext();)=0A=
			{=0A=
				sb.append(iter.next()).append(";\r\n");=0A=
			}=0A=
			throw new StepFailedException(sb.toString(), this);=0A=
		}=0A=
=0A=
		try=0A=
		{=0A=
			WebImage[] tabImg =3D fLastResponse.getImages();=0A=
			System.out.println(=0A=
				"tabImg: " + tabImg + " -> " + tabImg.length + " images");=0A=
			for (int i =3D 0; i < tabImg.length; ++i)=0A=
			{=0A=
				System.out.println(tabImg[i].getRequest().getURL());=0A=
				WebResponse resp =3D=0A=
					conversation.sendRequest(tabImg[i].getRequest());=0A=
			}=0A=
			if (true)=0A=
				throw new StepFailedException(=0A=
					getStepId(context) + ": missing images: " + tabImg,=0A=
					this);=0A=
			StringBuffer sb =3D new StringBuffer();=0A=
		}=0A=
		catch (Exception e)=0A=
		{=0A=
			throw new StepExecutionException(=0A=
				"Exception occured: " + e.toString());=0A=
		}=0A=
=0A=
		checkVisits(conversation, fLastResponse);=0A=
		if (fBrokenURLs.length() > 0)=0A=
		{=0A=
			throw new StepFailedException(=0A=
				getStepId(context)=0A=
					+ ": broken Link(s) : "=0A=
					+ fBrokenURLs.toString(),=0A=
				this);=0A=
		}=0A=
	}=0A=
=0A=
	/**=0A=
	* Checks the images and return the url of the missing ones=0A=
	*/=0A=
	protected List checkImages(=0A=
		WebConversation _conversation,=0A=
		WebResponse _response)=0A=
	{=0A=
		List liRep =3D new ArrayList();=0A=
		WebImage[] tabImg =3D null;=0A=
		try=0A=
		{=0A=
			tabImg =3D _response.getImages();=0A=
		}=0A=
		catch (SAXException e)=0A=
		{=0A=
			throw new StepExecutionException(=0A=
				"Exception occured: " + e.toString());=0A=
		}=0A=
=0A=
		for (int i =3D 0; i < tabImg.length; ++i)=0A=
		{=0A=
			try=0A=
			{=0A=
				WebResponse resp =3D=0A=
					_conversation.sendRequest(tabImg[i].getRequest());=0A=
			}=0A=
			catch (Exception e)=0A=
			{=0A=
				liRep.add(tabImg[i].getSource());=0A=
			}=0A=
		}=0A=
=0A=
		return liRep;=0A=
	}=0A=
=0A=
	public void checkVisits(WebConversation conversation, WebResponse =
response)=0A=
	{=0A=
		Visit[] visits =3D new Visit[0];=0A=
		try=0A=
		{=0A=
			visits =3D getVisits(conversation, response);=0A=
			storeBrokenUrls(visits, response);=0A=
		}=0A=
		catch (Exception e)=0A=
		{=0A=
			handleUnexpectedException(e);=0A=
		}=0A=
	}=0A=
=0A=
	protected void storeBrokenUrls(Visit[] visits, WebResponse response)=0A=
		throws MalformedURLException=0A=
	{=0A=
		for (int i =3D 0; i < visits.length; i++)=0A=
		{=0A=
			if (visits[i].isNotBroken() =3D=3D false)=0A=
			{=0A=
				fBrokenURLs.append(visits[i].getRequest().getURL().toString());=0A=
				fBrokenURLs.append(" on ");=0A=
				fBrokenURLs.append(response.getURL().toString());=0A=
				fBrokenURLs.append("; ");=0A=
			}=0A=
		}=0A=
	}=0A=
=0A=
	public int getLinkCount(WebResponse response)=0A=
		throws org.xml.sax.SAXException=0A=
	{=0A=
		return getGoodLinks(response).length;=0A=
	}=0A=
=0A=
	private WebLink[] getGoodLinks(WebResponse response)=0A=
		throws org.xml.sax.SAXException=0A=
	{=0A=
		WebLink[] links =3D response.getLinks();=0A=
		Vector linkVector =3D new Vector();=0A=
=0A=
		for (int i =3D 0; i < links.length; i++)=0A=
		{=0A=
			try=0A=
			{=0A=
				if (links[i]=0A=
					.getRequest()=0A=
					.getURL()=0A=
					.getProtocol()=0A=
					.equals("http"))=0A=
				{=0A=
					linkVector.addElement(links[i]);=0A=
				}=0A=
			}=0A=
			catch (MalformedURLException e)=0A=
			{=0A=
			}=0A=
		}=0A=
=0A=
		WebLink[] goodLinks =3D new WebLink[linkVector.size()];=0A=
		for (int j =3D 0; j < linkVector.size(); j++)=0A=
		{=0A=
			goodLinks[j] =3D (WebLink) linkVector.elementAt(j);=0A=
		}=0A=
		return goodLinks;=0A=
	}=0A=
=0A=
	public WebRequest[] getRequests(WebResponse response) throws =
SAXException=0A=
	{=0A=
		WebLink[] links =3D getGoodLinks(response);=0A=
		WebRequest[] requests =3D new WebRequest[links.length];=0A=
		for (int i =3D 0; i < requests.length; i++)=0A=
		{=0A=
			requests[i] =3D links[i].getRequest();=0A=
		}=0A=
		return requests;=0A=
	}=0A=
=0A=
	public Visit[] getVisits(=0A=
		WebConversation conversation,=0A=
		WebResponse response)=0A=
		throws MalformedURLException, SAXException=0A=
	{=0A=
		WebRequest[] requests =3D getRequests(response);=0A=
		List collector =3D new LinkedList();=0A=
		for (int i =3D 0; i < requests.length; i++)=0A=
		{=0A=
			WebRequest request =3D requests[i];=0A=
			if (!visited(request, collector))=0A=
			{=0A=
				collector.add(visit(request, conversation));=0A=
			}=0A=
		}=0A=
		return (Visit[]) collector.toArray(new Visit[collector.size()]);=0A=
	}=0A=
=0A=
	//todo: better use a Set and have equals() and hashcode() in Visit go =
to getRequest().getURL().toExternalForm()=0A=
	private boolean visited(WebRequest request, List visits)=0A=
		throws MalformedURLException=0A=
	{=0A=
		for (Iterator iterator =3D visits.iterator(); iterator.hasNext();)=0A=
		{=0A=
			Visit visit =3D (Visit) iterator.next();=0A=
			if (request=0A=
				.getURL()=0A=
				.toExternalForm()=0A=
				.equals(visit.getRequest().getURL().toExternalForm()))=0A=
			{=0A=
				return true;=0A=
			}=0A=
		}=0A=
		return false;=0A=
	}=0A=
=0A=
	Visit visit(WebRequest request, WebConversation conversation)=0A=
		throws MalformedURLException, SAXException=0A=
	{=0A=
		Visit visitResult =3D new Visit(request);=0A=
		visitResult.setNotBroken(false);=0A=
		try=0A=
		{=0A=
			logText(fContext, request.getURL().toExternalForm());=0A=
			WebResponse response =3D conversation.getResponse(request);=0A=
			visitResult.setResponse(response);=0A=
			visitResult.setNotBroken(response.getResponseCode() < 400);=0A=
		}=0A=
		catch (SAXException e)=0A=
		{=0A=
			visitResult.setNotBroken(true);=0A=
		}=0A=
		catch (HttpException e)=0A=
		{=0A=
			visitResult.setNotBroken(false);=0A=
		}=0A=
		catch (IOException e)=0A=
		{=0A=
			visitResult.setNotBroken(false);=0A=
		}=0A=
=0A=
		followRecursively(visitResult, conversation);=0A=
=0A=
		return visitResult;=0A=
	}=0A=
=0A=
	protected void followRecursively(=0A=
		Visit visitResult,=0A=
		WebConversation conversation)=0A=
		throws MalformedURLException, SAXException=0A=
	{=0A=
		if (visitResult.isNotBroken() && fCurrentDepth < fMaxDepth)=0A=
		{=0A=
			if (stopHunting(visitResult) =3D=3D false)=0A=
			{=0A=
				fCurrentDepth++;=0A=
				checkVisits(conversation, visitResult.getResponse());=0A=
				fCurrentDepth--;=0A=
			}=0A=
		}=0A=
	}=0A=
=0A=
	protected boolean stopHunting(Visit visitResult)=0A=
	{=0A=
		if (fOnSiteOnly && foreignHost(visitResult))=0A=
			return true;=0A=
		if (noWebPage(visitResult))=0A=
			return true;=0A=
		return false;=0A=
	}=0A=
=0A=
	protected boolean noWebPage(Visit visitResult)=0A=
	{=0A=
		return visitResult.getResponse().getContentType().indexOf("html") < 0;=0A=
	}=0A=
=0A=
	protected boolean foreignHost(Visit visitResult)=0A=
	{=0A=
		return visitResult.getResponse().getURL().getHost().equals(=0A=
			getBaseResponse().getURL().getHost())=0A=
			=3D=3D false;=0A=
	}=0A=
=0A=
	public WebResponse getBaseResponse()=0A=
	{=0A=
		return fLastResponse;=0A=
	}=0A=
=0A=
	//todo: go for int ?=0A=
	public void setDepth(String depth)=0A=
	{=0A=
		//dk: check for validity=0A=
		fMaxDepth =3D Integer.parseInt(depth);=0A=
	}=0A=
=0A=
	public void setOnSiteOnly(String onSiteOnly)=0A=
	{=0A=
		// todo: maybe call generic boolean handling=0A=
		if (onSiteOnly.equals("true"))=0A=
			fOnSiteOnly =3D true;=0A=
	}=0A=
=0A=
}
------=_NextPart_000_013A_01C32201.E2073260--