[Webtest] Dynamic field naming

Fri, 03 May 2002 15:04:48 -0700

Dierk Konig wrote:
 > While functional tests like WebTests are about the externally visible
 > behaviour of your system, unit tests are about the internal logic,
 > program design, interface conditions and communication between 
developers.
 > They enable the development team to refactor continously. The 
internal units
 > cover much more logic than what the user normally cares about
 > (correct handling of DB error conditions, correct construction of dynamic
 > SQL statements, proper implementation of loops, recursions, lookups, and
 > the like).
 >
 > Well unit tests really play in a different ballpark.
 >

Dierk, Erik, et.al.
    Since this started with something from my thread about 
unit/functional testing, I'll have to throw in Tim and my $.04.  Sorry 
about the length, but this is a philisophical discussion, not a 
technical one :-)
    True enough, unit and functional tests are indeed very different. 
Let's add a third term though:  acceptance tests!  The definition would 
be something like: "tests that a customer defines in order to indicate 
that they will accept the system as complete once they all pass".
    In Extreme Programming these acceptance tests are usually defined to 
correspond to stories (short descriptions/reminders) the customer has 
written about individual pieces of required functionality.  With this 
approach, a specification of a system is a large collection of stories 
on index cards, conversations, and customer-defined acceptance tests 
instead of outdated documents that are expensive to produce and 
maintain.  Once the development team has made all of the acceptance 
tests pass, the system is considered done (for the moment, anyway, until 
the customer adds or changes tests).
    These acceptance tests consistently seem to cross back and forth 
over the line between unit and functional tests because the customer 
cares about testing both the logic and the assembly of components.  They 
are usually testing from the user's perspective, doing operations like 
logging in and running a couple searches, or trying to add records. 
Naturally, from that perspective, they want to ensure the user sees the 
right data, as well as that it shows up in the right place.
    For most people doing Exreme Programming now this takes the form of 
the customer defining test steps in a spreadsheet or other document, and 
programmers writing quite boring source code from it.  Quite a waste of 
time, don't you think?  What would be best is if we could empower them 
to manage the tree of test specifications and test data, and change them 
when their needs change with very little training.  In order for that to 
be a real win, they need to be able to define them in an executable 
form, so that the developers could spend their time and the customer's 
money developing unit tests, functional tests, and source code for the 
system, not copying test specs from English (or German) into source.
    We originally thought that Webtest might be the tool to use in 
defining acceptance tests (for a web application, anyway), but it isn't. 
Webtest does what it set out to do quite nicely!  However, it was 
clearly built as a _functional_ testing tool for _developers_ to use, 
while what is needed for Extreme Programming in the web environment is 
an _acceptance_ testing tool for _customers_ to use.
    It seems that Herr Ostermueller is also discovering that his needs 
are more along the lines of an acceptance testing tool, as opposed to a 
functional testing tool, since he wants to delegate some of the testing 
to non-programmers.  Erik, correct me if I'm wrong, but I suspect that 
you want to enable these non-programmers to define tests as simple use 
cases that verify whatever they feel is important!  Since they aren't 
programmers, they don't know or care about the distinction between 
functional and unit tests, so what they feel is important will likely 
include a lot of logic testing.
    Clearly, this is all just opinion, but I think you'll find that the 
distinction between "functional" and "acceptance" tests is a useful one. 
   In the case of WebTest, this distinction helps illuminate why it 
doesn't fully meet our needs, and why we'd like to have access to more 
of HttpUnit's interface.
    Attempting to build an "AntHttpUnit" for acceptance testing, would, 
we believe, require a somewhat different outlook from the beginning, and 
most importantly, non-programmers helping to define the schema.
    So, that said, what do people think?  Is this distinction 
(acceptance/functional) helpful to you?  What would be useful in an 
acceptance testing tool?

      Ben && Tim


Dierk König wrote:
 > Hi Eric,
 >
 >
 >>>>Logic is to be handled by Unit Tests. (cf <http://www.junit.org>)
 >>>
 >>Does anyone have a more specific reference?
 >
 >
 > Behind the link is the best collection of information about unit testing.
 > You may want to refer to the "articles" section for intoduction.
 >
 > For the topic of logic and unit testing I don't disagree with your view.
 >
 > It seems you have a number of testers, developers and non-developers, 
that
 > are capable of writing webtests. That is great! Make your users define
 > their expectations as written tests and you will have a huge benefit.
 >
 > My statement about "logic" (maybe a bad wording) was based on some 
feature
 > requests that would extend WebTest into a full-blown programming 
language.
 > And it is my considered opinion that it needs a programmer to handle
 > a programming language. WebTest is very restricted as much as programming
 > logic is concerned. A WebTest is always linear in execution without any
 > complicated control flow. We believe this to be a benefit.
 >
 > BTW: for those who want to write WebTests using a full-blown programming
 > language, the can do so: just write your tests in Java using HttpUnit
 > directly.
 >
 > While functional tests like WebTests are about the externally visible
 > behaviour of your system, unit tests are about the internal logic,
 > program design, interface conditions and communication between 
developers.
 > They enable the development team to refactor continously. The 
internal units
 > cover much more logic than what the user normally cares about
 > (correct handling of DB error conditions, correct construction of dynamic
 > SQL statements, proper implementation of loops, recursions, lookups, and
 > the like).
 >
 > Well unit tests really play in a different ballpark.
 >
 > Functional tests are like drawing samples from the countless possible
 > combinations.
 >
 > Well, I just found that I'm not able to tell the whole difference in
 > a mail. I can only point yout to the resources and try to make you
 > curious about trying to use unit testing yourself.
 >
 > cheers
 > Mittie
 >
 > BTW: You will find that WebTest itself has a number of functional tests
 > but also a reasonable number of unit tests. Maybe the WebTest sources 
give
 > some hints on where the differences are.
 >
 >
 >
 >>-----Original Message-----
 >>From: webtest-admin@lists.canoo.com
 >>[mailto:webtest-admin@lists.canoo.com]On Behalf Of
 >>Erik.Ostermueller@alltel.com
 >>Sent: Mittwoch, 1. Mai 2002 20:57
 >>To: webtest@lists.canoo.com
 >>Subject: [Webtest] Dynamic field naming
 >>
 >>
 >>First, thanks for providing the community with such a great product.
 >>
 >>I'd like to revisit a thread that ended with this email:
 >>http://lists.canoo.com/pipermail/webtest/2002q1/000038.html
 >>
 >>Dierk Konig wrote:
 >>
 >>>>Logic is to be handled by Unit Tests. (cf <http://www.junit.org>)
 >>>
 >>Does anyone have a more specific reference?  I looked and could not find
 >>it.
 >>If one assumes that logic is only for unit tests, then that suggests
 >>that only developers can test logic.  Is this a correct assumtion?  This
 >>doesn't seem right to me.  We have a great number of non-developers in
 >>our organization that are capable of using XML files to test the logic
 >>of the system.  We want to share the ownship of testing between
 >>developers and non-developers.  That is tough to do when logic testing
 >>is encoded in source code.
 >>
 >>
 >>
 >>_______________________________________________
 >>WebTest mailing list
 >>WebTest@lists.canoo.com
 >>http://lists.canoo.com/mailman/listinfo/webtest
 >>
 >
 >
 > _______________________________________________
 > WebTest mailing list
 > WebTest@lists.canoo.com
 > http://lists.canoo.com/mailman/listinfo/webtest
 >
 >


-- 
Truth is within ourselves; it takes no rise
  From outward things, whate're you may believe
There is an inmost center in us all,
Where Truth abides in fulness;...and, to KNOW,
Rather consists in opening out a way
Whence the imprisoned splendor may escape,
Than in effecting entry for a light
Supposed to be without
    --  Robert Browning, "Paracelsus"