[Webtest] HTTP error 302
John and Pip
John and Pip" <pipandjohn@gmail.com
Wed, 3 Jan 2007 09:37:49 +1300
Hi Yet Again,
The relevant code in HTMLUnit seems to be in WebClient.java:
if( statusCode >=3D 301 && statusCode <=3D307 && isRedirectEnabled(=
) ) {
URL newUrl =3D null;
String locationString =3D null;
try {
locationString =3D webResponse.getResponseHeaderValue("Loca=
tion");
newUrl =3D expandUrl( fixedUrl, locationString);
}
catch( final MalformedURLException e ) {
getLog().warn("Got a redirect status code ["+statusCode+" "
+webResponse.getStatusMessage()
+"] but the location is not a valid url
["+locationString+"]. Skipping redirection processing.");
return webResponse;
}
getLog().debug("Got a redirect status code ["+statusCode
+"] new location=3D["+locationString+"]");
if (webRequestSettings.getSubmitMethod().equals(SubmitMethod.GE=
T)
&&
webResponse.getUrl().toExternalForm().equals(locationString) ) {
getLog().warn("Got a redirect but the location is the
same as the page we just loaded ["
+ locationString + "]. Skipping redirection.");
}
else if ((statusCode =3D=3D 301 || statusCode =3D=3D 307)
&& method.equals(SubmitMethod.GET) ) {
final WebRequestSettings wrs =3D new
WebRequestSettings(webRequestSettings, newUrl);
wrs.setRequestParameters(parameters);
return loadWebResponse(wrs);
}
else if (statusCode <=3D 303) {
final WebRequestSettings wrs =3D new
WebRequestSettings(webRequestSettings, newUrl);
wrs.setSubmitMethod(SubmitMethod.GET);
return loadWebResponse(wrs);
}
It looks like a simple change could fix this, but not being a java
programmer I'm not all that confident. Can anyone comment as to
whether it makes sense for a GET redirect to the same page to not be
followed, or whether it would make more sense to allow it, and hence
make a fix to HTMLUnit code above? My experience suggests the latter.
If anyone could patch it and send to me I'd be most grateful,
otherwise I'll battle through and see how I go!
regards,
John
On 1/3/07, John and Pip <pipandjohn@gmail.com> wrote:
> Hi Again,
>
> I have now tried loading these pages using Tamper Data to try to look
> at what happens.
>
> https://flightbookings.airnewzealand.co.nz/isbook_en_NZ?review?initValida=
tePNR.do
>
> returns a 302 error and then I get
>
> https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSearchFo=
rFlights.do?review?initValidatePNR.do
>
> which returns a 200 code.
>
> So I will try to track down the app developers and find out what is going=
on.
>
> But, I still don't understand why WebTest can't handle the proper URL:
>
> https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/review/initValida=
tePNR.do
>
> When I look at this in Tamper Data it shows the first load of this
> page giving a 302 error, then a second load of the same page giving a
> 200 result.
>
> So I think I was on the right track below that WebTest is failing
> because it can't handle the same page being reloaded as per this:
>
> [invoke] DEBUG (com.gargoylesoftware.htmlunit.WebClient) - Got a
> redirect status code [302] new
> location=3D[https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/revie=
w/initValidatePNR.do]
> [invoke] WARN (com.gargoylesoftware.htmlunit.WebClient) - Got a
> redirect but the location is the same as the page we just loaded
> [https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/review/initValid=
atePNR.do].
> Skipping redirection.
> [invoke] INFO (com.gargoylesoftware.htmlunit.WebClient) -
> statusCode=3D[302] contentType=3D[text/html]
> ...
> [invoke] DEBUG (com.canoo.webtest.steps.Step) - Step failed at
> C:\WebTest\doc\ISIS\testTest.xml:9: with exception
> [invoke] com.canoo.webtest.engine.StepFailedException: Step[invoke
> "Get the ISIS retrieve booking page" (1/1)]: HTTP error 302, at:
> invoke, Step: InvokePage at C:\WebTest\doc\ISIS\testTest.xml:9: with
> (resultFilename=3D"response_1167767144056_invoke.html",
> taskName=3D"invoke")
>
> I tried using Canoo Webtest: R_1438 then with Canoo Webtest: R_1454
> with same result.
>
> Is this a defect in WebTest that I should raise in JIRA and is there a
> workaround? Any help much appreciated!
>
> regards,
> John
>
> On 1/3/07, John and Pip <pipandjohn@gmail.com> wrote:
> > Hi,
> >
> > I posted some months back about HTTP 302 errors, and in the thread
> > below (snipped down somewhat) was given advise to use ? rather than /
> > in the URL so I changed it to :
> > url=3D"https://flightbookings.airnewzealand.co.nz?isbook_en_NZ?book?ini=
tSearchForFlights.do
> > and it worked.
> >
> > However, I now need to be able to hit a second "start page" in the
> > same web application. That is the URL:
> > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/review/initVali=
datePNR.do
> > This is a public URL and if you hit it you get a page headed up
> > "Retrieve online booking" and with a page title starting "Air New
> > Zealand Bookings - Retrieve Booking".
> >
> > This gives the same 302 error in WebTest.
> >
> > <failure message=3D"com.canoo.webtest.engine.StepFailedException:
> > Step[invoke "Get the ISIS retrieve booking page" (1/1)]:
> > HTTP error 302, at: invoke, Step: InvokePage at
> > C:\WebTest\doc\ISIS\testTest.xml:6: with
> > (resultFilename=3D"response_1167729673189_invoke.html",
> > taskName=3D"invoke")"/>
> >
> > At warn level the console shows:
> >
> > [invoke] WARN (com.gargoylesoftware.htmlunit.WebClient) - Got a
> > redirect but the location is the same as the page we just loaded
> > [https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/review/initVal=
idatePNR.do].
> > Skipping redirection.
> >
> > Which perhaps is the clue - does this site require a redirection to
> > the same URL, and if so is there any way to make WebTest / HTMLUnit
> > *not* ignore this?
> >
> > However, when I follow the same trick of changing / to ? as per:
> >
> > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ?review?initVali=
datePNR.do
> >
> > I get taken to a completely different page - specifically to
> > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSearch=
ForFlights.do
> >
> > The same thing happens if I put that URL directly into a browser so
> > I'm just completely confused. Can anyone shed any further light on
> > this for me please?
> >
> > thanks and regards,
> > John
> >
> > Here is the script:
> >
> > <webtest name=3D"Retrieve PNR" description=3D"no desc yet" >
> > &config;
> > <steps>
> > <invoke
> > url=3D"https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/review/i=
nitValidatePNR.do"/>
> > </steps>
> > </webtest>
> >
> > and here is the config content:
> > <config
> > host=3D"registration.airnewzealand.com"
> > port=3D"443"
> > protocol=3D"https"
> > basepath=3D""
> > summary=3D"true"
> > saveresponse=3D"true"
> > resultpath=3D"..\webtest-results"
> > resultfile=3D"results.xml"
> > showhtmlparseroutput=3D"false"
> > autorefresh=3D"true"
> > haltonfailure=3D"false"
> > timeout=3D"120">
> > <option name=3D"RedirectEnabled" value=3D"true" =
/>
> > </config>
> >
> > I had appended the full debug level output but it exceeded the 40k
> > limit so have omitted it.
> >
> > On 11/18/06, John and Pip <pipandjohn@gmail.com> wrote:
> > > Hi Aarathi and Paul,
> > >
> > > I tried it and it works. I'm kind of amazed at the apparent black art=
s
> > > involved here - is it some sort of strange handling of pseudo-directo=
ry
> > > paths in URLS? I notice the URL with the ?s works directly into a bro=
wser as
> > > well.
> > >
> > > But I don't understand your comment paul. I wanted it to follow redir=
ects so
> > > if default behaviour is to do so why do I need to replace / with ? in=
the
> > > URLs?
> > >
> > > Thanks and regards,
> > > John
> > >
> > >
> > > On 11/17/06, Paul King <paulk@asert.com.au> wrote:
> > > > Rob Manthey wrote:
> > > > > ok, a moment to learn something ... is this simply the final url =
(after
> > > > > the 302 redirect) or is there something else going on here?
> > > >
> > > > The default behaviour is to follow redirects. If you want to visit
> > > > the 302 page you have to turn redirects off among other things.
> > > > I guess we should document this better as it arises quite frequentl=
y.
> > > >
> > > > Cheers,
> > > >
> > > > Paul.
> > > >
> > > > > on 17/11/06 11:49 Aarathi Niranjan said the following:
> > > > >>
> > > > >> Just a suggestion =85 this works for me =85 try it:
> > > > >>
> > > > >>
> > > > >>
> > > > >> =85
> > > > >>
> > > > >> <steps>
> > > > >>
> > > > >> <invoke description=3D"ISIS SFF NZ searc=
h page"
> > > > >>
> > > url=3D"https://flightbookings.airnewzealand.co.nz?isbook_en_NZ?book?i=
nitSearchForFlights.do
> > > "
> > > > >> />
> > > > >>
> > > > >> </steps>
> > > > >>
> > > > >> =85
> > > > >>
> > > > >>
> > > > >>
> > > > >> Hope that helps.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > ---------------------------------------------------------------------=
---
> > > > >>
> > > > >> *From:* webtest-admin@lists.canoo.com
> > > > >> [mailto: webtest-admin@lists.canoo.com] *On Behalf Of *Charlie K=
uharski
> > > > >> *Sent:* Thursday, November 16, 2006 2:50 PM
> > > > >> *To:* webtest@gate2.canoo.com
> > > > >> *Subject:* Re: [Webtest] HTTP error 302
> > > > >>
> > > > >>
> > > > >>
> > > > >> Reaching via a browser is fine. I can see it also. Know that W=
ebTest
> > > > >> is build upon the HtmlUnit's API.
> > > > >> As far as being reachable via HtmlUnit, this is only a guess, bu=
t I'm
> > > > >> guessing the URL is passed through to be used by HtmlUnit. So I=
'm
> > > > >> guess it's the culprit. There is a slight chance it's WebTest
> > > > >> mangling the name somehow, but I doubt it. Wouldn't know for su=
re
> > > > >> until someone looks at the source.
> > > > >> Try using the IP address instead.
> > > > >>
> > > > >> John and Pip wrote:
> > > > >>
> > > > >> Hi Charlie,
> > > > >>
> > > > >> Thanks for your response.
> > > > >>
> > > > >> Do you mean you couldn't reach
> > > > >>
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do
> > > > >> from a browser or that you think WebTest believes it is not reac=
hable?
> > > > >> This is the main public URL of the Air New Zealand flight bookin=
g
> > > > >> engine, and we have a screen up in the office showing bookings c=
oming
> > > > >> through every minute!
> > > > >>
> > > > >> re testing with HTMLUnit: I'm not a java programmer - just a tes=
ter,
> > > > >> trying to figure out whether I need to be a java programmer to u=
se
> > > > >> WebTest!
> > > > >>
> > > > >> Thanks for the hint re version number. I see this is R_1389. In =
fact
> > > > >> to my embarrassment I see this even appears in my email below!
> > > > >>
> > > > >> regards,
> > > > >> John
> > > > >>
> > > > >> On 11/17/06, *Charlie Kuharski* <ckuharsk@nmdp.org
> > > > >> <mailto: ckuharsk@nmdp.org>> wrote:
> > > > >>
> > > > >> Your host is not reachable. I'm guessing it HtmlUnit. Try usin=
g
> > > > >> something without the "...co.nz". You could try and test with Ht=
mlUnit.
> > > > >> An ugly was to discover the version is to
> > > > >>
> > > > >> 1) replace your steps with this only:
> > > > >> <verifyTitle
> > > > >> description=3D"we should see the login title"
> > > > >> text=3D"Login Page" />
> > > > >>
> > > > >> 2) you should see something like:
> > > > >> C:\test\Translink\gtest1.xml:12: Canoo Webtest:
> > > R_1393.
> > > > >> Test failed.
> > > > >> Exception raised:
> > > com.canoo.webtest.engine.StepExecutionException : No
> > > > >> current response available! Is previous invoke miss
> > > > >> itle at C:\test\Translink\gtest1.xml:30: with
> > > (description=3D"we should
> > > > >> see the login title", taskName=3D"verifyTitle", text
> > > > >> anoo.webtest.engine.StepExecutionException: No current
> > > response
> > > > >> available! Is previous invoke missing?, Step: VerifyTitle
> > > > >> nk\gtest1.xml:30: with (description=3D"we should see the login =
title",
> > > > >> taskName=3D"verifyTitle", text=3D"Login Page")
> > > > >>
> > > > >> Good luck.
> > > > >>
> > > > >>
> > > > >>
> > > > >> John and Pip wrote:
> > > > >>
> > > > >> Hi,
> > > > >>
> > > > >> I have a test that is failing to follow a 302 error. I've search=
ed the
> > > > >> archives and played around trying to enable redirects but nothin=
g
> > > > >> seems to work.
> > > > >>
> > > > >> Below is the stripped-down test (just one invoke) and the output=
. This
> > > > >> is a publicly available page so you can hit
> > > > >>
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do
> > > > >> from either IE or Firefox so I'm hoping someone can help me figu=
re out
> > > > >> how to hit this page from WebTest! Note that I've tried an inter=
nal
> > > > >> page that is http rather than https and get the same problem so =
it
> > > > >> isn't related to SSL.
> > > > >>
> > > > >> I am using Java 1.5.0.6 < http://1.5.0.6>, HTMLUnit 1.10 and Web=
Test
> > > > >> R_1393 (I think! How can I verify this?).
> > > > >>
> > > > >> Regards,
> > > > >>
> > > > >> John
> > > > >>
> > > > >> <?xml version=3D"1.0"?>
> > > > >>
> > > > >> <!DOCTYPE project SYSTEM "WebTest.dtd">
> > > > >>
> > > > >> <project name=3D"OLH" basedir=3D"." default=3D"main">
> > > > >>
> > > > >> <property name=3D"webtest.home" location=3D"..\.."/>
> > > > >>
> > > > >> <import file=3D"${ webtest.home}/lib/taskdef.xml"/>
> > > > >>
> > > > >> <target name=3D"main">
> > > > >>
> > > > >> <webtest name=3D"ISIS SFF NZ Search">
> > > > >>
> > > > >> <steps>
> > > > >>
> > > > >>
> > > <invoke
> > > > >> description=3D"ISIS SFF NZ search page" url=3D"
> > > > >>
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do"
> > > > >> />
> > > > >>
> > > > >> </steps>
> > > > >>
> > > > >> </webtest>
> > > > >>
> > > > >> </target>
> > > > >>
> > > > >> </project>
> > > > >>
> > > > >>
> > > > >>
> > > > >> Here is the output:
> > > > >>
> > > > >>
> > > > >>
> > > > >> <?xml version=3D" 1.0" encoding=3D"UTF-8"?>
> > > > >>
> > > > >> <summary Implementation-Title=3D"Canoo Webtest"
> > > > >> Implementation-Version=3D"R_1389">
> > > > >>
> > > > >> <testresult endtime=3D"Fri Nov 17 09:25:42 NZDT 2006"
> > > > >>
> > > > >> location=3D"F:\WebTest\doc\ISIS\tempTest.xml:7:
> > > " starttime=3D"Fri
> > > > >> Nov 17 09:25:40 NZDT 2006"
> > > > >>
> > > > >> successful=3D"no" testspecname=3D"ISIS SFF NZ Search">
> > > > >>
> > > > >> <config>
> > > > >>
> > > > >> [snipped because I give the full URL below =96 get s=
ame
> > > > >> result if set up config properly including port 443]
> > > > >>
> > > > >> </config>
> > > > >>
> > > > >> <results>
> > > > >>
> > > > >> <step>
> > > > >>
> > > > >> <parameter name=3D"description" value=3D"ISIS SF=
F NZ
> > > > >> search page"/>
> > > > >>
> > > > >> <parameter name=3D"method" value=3D"GET"/>
> > > > >>
> > > > >> <parameter name=3D"resultFilename"
> > > > >> value=3D"response_1163708742276_invoke.html"/>
> > > > >>
> > > > >> <parameter name=3D"taskName" value=3D"invoke"/>
> > > > >>
> > > > >> <parameter name=3D"url"
> > > > >> value=3D"
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do
> > > > >> <
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do
> > > >"/>
> > > > >>
> > > > >>
> > > > >> <result>
> > > > >>
> > > > >> <failed duration=3D"1469"/>
> > > > >>
> > > > >> </result>
> > > > >>
> > > > >> <htmlparser>
> > > > >>
> > > > >> <error col=3D"1" line=3D"1" url=3D"
> > > > >>
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do">Empty
> > > > >> document.</error>
> > > > >>
> > > > >> </htmlparser>
> > > > >>
> > > > >> </step>
> > > > >>
> > > > >> <failure
> > > > >> message=3D"com.canoo.webtest.engine.StepFailedException:
> > > Step[invoke
> > > > >> "ISIS SFF NZ search page" (1/1)]: HTTP error 302, at:
> > > > >> invoke, Step: InvokePage at
> > > F:\WebTest\doc\ISIS\tempTest.xml:9: with
> > > > >> (description=3D"ISIS SFF NZ search page",
> > > > >> method=3D"GET",
> > > > >>
> > > resultFilename=3D"response_1163708742276_invoke.html",
> > > > >> taskName=3D"invoke",
> > > > >>
> > > url=3D"https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/b=
ook/initSearchForFlights.do"
> > > > >> <
> > > https://flightbookings.airnewzealand.co.nz/isbook_en_NZ/book/initSear=
chForFlights.do">;)"/>
> > > > >>
> > > > >> </results>
> > > > >>
> > > > >> </testresult>
> > > > >>
> > > > >> </summary>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > >
> > > > _______________________________________________
> > > > WebTest mailing list
> > > > WebTest@lists.canoo.com
> > > > http://lists.canoo.com/mailman/listinfo/webtest
> > > >
> > >
> > >
> >
>