Why do you use WebTest? ;-)
|
View:
New views
6 Messages
—
Rating Filter:
Alert me
|
 |
Why do you use WebTest? ;-)
by Marc Guillemot
::
Rate this Message:
Hi all,
why do you use WebTest rather than some other test tool (commercial or open source)? Of course I have some ideas on this question ;-) but as WebTest committer I'm surely not objective when comparing WebTest to others. While presenting WebTest I've already encountered difficulties to convince that it was the tool of choice. I think that it occurs particularly when people don't have any real experience of automated web application tests and therefore can't understand "real" arguments. From your experience as WebTest user, what would you say to persuade someone wanting to start test automation for a webapp to use WebTest rather than something else? Marc. _______________________________________________ WebTest mailing list WebTest@... http://lists.canoo.com/mailman/listinfo/webtest |
|
|
Re: Why do you use WebTest? ;-)
by John and Pip
::
Rate this Message:
Hi Marc,
Jury is still out as to whether we will actually end up using it, but the reasons it is one of two I'm evaluting is: - Ease of install, upgrade, maintenance compared to the Big4 tools such as Rational. - Cost = $0 - Support = good and $0 - Advantages of declarative XML format but with addition of good scripting support (that would probably let us code the action-word framework in Groovy if we needed to - see below) - Open architecture - Support parameters to suite e.g. environment or base URL as well as to test case and step. - Support nested tests and suites - Support compex named data elements e.g. person that can be retrieved from separate "file" to the test steps and simple data elements. Areas where it is not bad but perhaps some others are better are: - Handle IE and Firefox. WebTest does not drive a real browser. We need pages that work in IE and Firefox so ideally in our web-gui-based testing we would be testing this, not functionality in the simulated environment of HTMLUnit. One example of this is having to put ? in path of URL instead of / to avoid 302 errors that I posted about last week. Another example is the confirmation page not being found that I posted about this week. - Handling of JavaScript. Problem here is if we have a page that has javascript that makes HTMLUnit spew, but that page uses a javascript link to move to next page in flow, my end to end test dies - Can be set to log and to produce screenshots esp. on error. This sort of works I guess but without the stylesheet references. - Must not break if page contains ActiveX, Flash, .Net constructs and whatever else. I need a tool that doesn't force me to ask developers to change code. - Handles AJAX - Can read input files and produce output files in one of and ideally all of csv, Excel, XML. This can probably be handled by Groovy where necessary. - No duplication so each part of system (usually each screen) needs just one fixture accessing it. Specifically we have been looking at keyword-driven, also called action-word, frameworks, and I'm trying to figure out how the rather different paradigm of WebTest maps to this, and reviewing in fact whether our asumptions that action-word is the way to go is correct. Ideally want just one "script" having to understand the implementation details of each page or chunk of a page. For the record, the comparison I'm using in terms of more traditional tools is QEngine from AdventNet. It is not free but at $700 US is a lot cheaper than the Big4 and has excellent support. I think WebTest is a great tool, and the developers have done a wonderful job. I think WebTest and Fit/Fitnesse represent the future of web gui functional testing. If WebTest could transparently select whether to use HTMLUnit or a real browser to execute the same test (not sure how - don't like selenium etc that require proxies so would I think be best to drive browser through their own native interface e.g. Com/ActiveX I guess for IE), and if it could be supported as a fixture within Fitnesse, that would be Nirvana for me. regards, John On 11/24/06, Marc Guillemot <mguillemot@...> wrote: Hi all, |
|
|
Re: Why do you use WebTest? ;-)
by Paul King
::
Rate this Message:
John and Pip wrote:
> - Handle IE and Firefox. WebTest does not drive a real browser. We need > pages that work in IE and Firefox so ideally in our web-gui-based > testing we would be testing this, not functionality in the simulated > environment of HTMLUnit. One example of this is having to put ? in path > of URL instead of / to avoid 302 errors that I posted about last week. > Another example is the confirmation page not being found that I posted > about this week. > > [..]. If WebTest could transparently select > whether to use HTMLUnit or a real browser to execute the same test (not > sure how - don't like selenium etc that require proxies so would I think > be best to drive browser through their own native interface e.g. > Com/ActiveX I guess for IE), and if it could be supported as a fixture > within Fitnesse, that would be Nirvana for me. We could consider embedding jwebunit 2.x but it sort of has lowest common denominator functionality which doesn't really work for me: http://jwebunit.sourceforge.net/2.x/index.html Paul. _______________________________________________ WebTest mailing list WebTest@... http://lists.canoo.com/mailman/listinfo/webtest |
|
|
|
|
|
RE: Why do you use WebTest? ;-)
by Colin O'Brien-2
::
Rate this Message:
Hi Marc et al,
For me (and CAPE) I guess the main thing is the open source status of WebTest, we don't have the time or resources to pour into an all encompassing solution like TestDirector or the Rational suite of test management applications. Secondly, I found that the learning curve with WebTest is a lot less steep than some other tools, once I was introduced to the tool I found myself writing useful test cases within a couple of hours... A third and very significant reason is that once the test cases have been written they can be executed in an unsupervised manner i.e. I can kick them off on my office machine and still be able to continue with other tasks on the same box. With other GUI tools, such as those of the record and playback genre, they take over the hardware resource and you cannot use the machine until the test suite has completed. The documentation and support available via the mailing list is also a significant factor in our continued use of WebTest. I have tried to contact some other mailing lists regarding initial startup or errors encountered with simple test cases and I have not received anywhere near the same level of knowledgeable support I have received from the WebTest community. Obviously the continued development and improvement of the tool is also another factor, prompt addition of new features and correction of issues is a huge bonus for users of the tool. Throw in the portable nature of the solution also so it can run on Win* or *NIX OS environments, there's another reason... Hopefully this goes someway towards answering your question... :-) Kind Regards, Colin. -----Original Message----- From: webtest-admin@... [mailto:webtest-admin@...] On Behalf Of Marc Guillemot Sent: 23 November 2006 15:44 To: webtest@... Subject: [Webtest] Why do you use WebTest? ;-) Hi all, why do you use WebTest rather than some other test tool (commercial or open source)? Of course I have some ideas on this question ;-) but as WebTest committer I'm surely not objective when comparing WebTest to others. While presenting WebTest I've already encountered difficulties to convince that it was the tool of choice. I think that it occurs particularly when people don't have any real experience of automated web application tests and therefore can't understand "real" arguments. From your experience as WebTest user, what would you say to persuade someone wanting to start test automation for a webapp to use WebTest rather than something else? Marc. _______________________________________________ WebTest mailing list WebTest@... http://lists.canoo.com/mailman/listinfo/webtest _______________________________________________ WebTest mailing list WebTest@... http://lists.canoo.com/mailman/listinfo/webtest |
|
|
Re: Why do you use WebTest? ;-)
by Lisa Crispin-2
::
Rate this Message:
We use a combination of tools for test automation. We could have purchased vendor tools, we chose the tools we have because they fit our needs.
The tools we use are: JUnit for programmer-facing tests (we practice TDD) FitNesse for customer-facing tests behind the GUI (and occasionally, programmer-facing tests, when it's easier to do TDD for a particular bit of code that way) WebTest for regression tests using the GUI WATIR for some GUI regression testing and scripts to set up for manual tests, where we needed a lot of flexibility and particular features (WebTest probably provides some of the features through using Groovy or similar, but for this it was easier for us to ramp up on Ruby) We chose WebTest 3 years ago when we started an agile development approach. We had no automated tests at all. We looked for a tool that was programmer-friendly, but easy for a tester with some technical skills to ramp up on, fairly independently. The fact that it integrates so easily with CruiseControl was another big 'selling' point. Within 8 months, we had a suite of smoke tests than covered all the critical parts of our app. This suite first ran as part of a nightly build, now it runs as part of a continual 'full' build that also runs FitNesse suites. At some point we had a lot of trouble with Javascript so we started using WATIR for scripts where we couldn't get WebTest to work. I think this was a couple years ago. My fellow tester was a Perl programmer so it was easy for him to get up to speed on Ruby and he wrote some nice scripts that accepted lots of command line parameters to set up particular data scenarios for manual testing. He preferred WATIR to WebTest, just a personal preference, so he tended to automate all his GUI tests there. We still used our extensive WebTest suite. The WATIR tests are very difficult and expensive to maintain. The tests have to be tested because they contain complex logic, whereas the WebTest scripts are specified and we rarely use any logic such as the ifStep. Also, we haven't gone to the trouble to try to integrate WATIR scripts with our build process (since it runs on Windows). At Agile 2006 I attended Paul King's WebTest tutorial and realized there were lots of robust new features. Meanwhile, the author of most of our WATIR scripts left and the maintenance had become a big burden. Everyone was having to learn Ruby (I enjoyed learning it myself, but it is difficult for me) and try to understand some complex code in order to work on the WATIR scripts. We looked back to WebTest and discovered all the Javascript issues we had with WebTest scripts in the past were solved, and we've been automating pretty much all of our GUI tests in WebTest since then. The tests are much quicker to write and cheap to maintain. We have to sharpen up some of our HTML coding but we should do that anyway. We will keep our WATIR script and maintain it, but I expect we will write most tests for new features in WebTest. We are very happy with our combination of tools covering our different needs. We're always open to new tools - a couple of the programmers saw Selenium at a conference and are eager to explore it more. But as the changes in WebTest have shown, good tools grow and change, and the team's needs or skill sets may change too, so it's important to always keep all options open. Our WebTest scripts, while quite simple, find regression bugs quite often. We don't have to change them often, and when we do (due to UI changes) it goes quickly. The ROI on them is tremendous. I've used many vendor tools successfully over the past 10 - 12 years, and the ROI on WebTest is many orders of magnitude higher than any tool I've used. Our programmers are able and willing to update scripts when necessary (with much less complaining than they do about the Ruby scripts) and our entire team is happy with it. -- Lisa On 11/23/06, Marc Guillemot <mguillemot@...> wrote: Hi all, -- Lisa Crispin Co-author, Testing Extreme Programming http://lisa.crispin.home.att.net |
| Free embeddable forum powered by Nabble | Forum Help |
