Having used the 1.0.3 version of Selenium RC for several weeks I now have an overall impression of the new release. The thing that bugged me most was that the upgrade was not as smooth as I hoped - several tests that previously worked fine with 1.0.1 started to fail with 1.0.3. We were able to relatively quickly find workarounds for what seem to be bugs introduced in the new release. Here are two links to the Selenium issue database in case you run into the same or similar problems - 400 and 408; look for 'XHR ERROR' exceptions in your logs when opening URLs.
With the workarounds our tests became as stable as before – not better and not worse. The new version did not really bring any noticeable improvements for me – I have not yet moved to Firefox 3.6 and I am not a Mac user. So my advise would be – If you have your tests running smoothly on 1.0.1 and you don't need Snow Leopard and/or FF 3.6 support yet, don't bother upgrading for now. There might be a new release on the horizon.
"Testers get indignant when they realize that the quality tools purchased at a serious price are themselves full of bugs. Indeed, test tools are often buggier than comparable (but cheaper) development tools. Plan to test your tools and spend time finding workarounds for bugs."
The quote comes from one of the best books on software quality I have read - Lessons Learned in Software Testing
by Cem Kaner, James Bach and Bret Pettichord. Be sure to read it if you haven't already.
I can not agree more with the authors. I spend a lot of my time in test automation trying to find workarounds for issues with the tool. Some of those issues can be considered 'functional idiosyncrasies', but most are just plain and simple bugs. During my work in the field of software quality I have noticed that users are normally quite tolerant of bugs, unless the bugs block them from doing their jobs. So for me there are two kinds of bugs in the test tools - those you can find a workaround for and those you can not.
If the tool is flexible enough and you know how to use it, most of the time you can make it do what you want - like for example using keyboard shortcuts and the clipboard to read the text from an unsupported GUI control. This is one advantage of tools that offer the flexibility of a real programming language for scripting.
There are also such things, for which you can not do much except waiting for the vendor to provide you a fix, and this can take long time. A few years ago I tried to upgrade to the latest TestPartner version (it was 5.2.3 back then). After the upgrade I found that all my automated scripts were causing random crashes in Internet Explorer. I just stayed with the old version until 5.6 came out, where this problem was fixed.