I have decided to sign up with Sky for their TV, broadband and phone package. This is mostly because I’m paying about as much for broadband and calls (mostly line rental actually) as Sky’s whole package. The Sky+ box really seals the deal though: no more trying to catch up with TV on BBC’s iPlayer (buffering……….) or having to muck about with DVD recorders, etc.

I wanted to enquire about what happened with line rental. The website seemed to handwave this detail, but you can’t just not explain an £11-£12/month charge or assume it’s a given. There appeared to be no contact number for them either, so I eventually got through to their sales dept via their SkyTalk (telephony services) number. Having said that their Dunfermline call centre handled the entire signup process very efficiently and courteously so full marks there.

I even have an installation appointment on a Sunday morning. That’s service.

So the automated signup/transfer process has now kicked in. This is where they fall down on detail.

Received a text this morning: “Your Sky Broadband will be activated on 13/01/2010.” Sorry, that’s impossible without me giving you the MAC key from my current broadband provider. There seems to be no mechanism by which I can send them this essential information. Maybe there’ll be a way via the member’s area on the website (MySky)? I got a confirmation email from them after signing up over the phone. Yep, the email has my username but no mention of a password! Maybe I gave them a password over the phone? Maybe it’s a randomised one you get when an account is created? No idea and they don’t hint at what’s going on.

Let’s try signing up without a password (you never know). Nope, can’t do that. OK, I’ll click the “Forgot password?” link even though I’ve not forgotten it; I’ve just never been told.

Email received with password which I can sign in with. Right, we’re moving again…

There’s no indication from the broadband setup progress page about what to do when migrating. Users this applies to who aren’t aware of this (assuming they’ve got this far) will get delayed because what they should be doing is contacting their current provider for their MAC key. A search through their help system does reveal the right page which does explain things clearly, but you have to know to look for it.

Moral: Join up your customer journeys so they are seamless. It’s no use having helpful staff on the phone if you then fall at the first few hurdles on the website.

Extra Moral: If you’re stopping your line, there’s no migration process (hence the redactions above). It magically works from the changeover point, which is a vast improvement on the old 7-10 day lead time for broadband activation.

How SQLite Is Tested is an extremely detailed account of how the team behind the world’s most popular embedded database verify its robustness.

It’s worth at least a skim read to see what they do. Some things jump out:

• Code required to comprehensively check other code is necessarily significantly larger and more complex than that which it’s testing. We already know that, but in SQLite’s case the testing code is 690 times larger than the source code.

• Regression testing must include specific checks for every bug ever reported, which are then used to show that the fixes hold for each release. Again, no surprises there, but the consequence is that this effort quickly snowballs into a lot of extra work and new releases take longer and longer to deliver.

• Static analysis has never proved useful (for them): not a single bug has been detected using this method. Even though this is mostly because their automated testing does such a thorough job it’s worth investigating how that might apply to your testing.

• They only achieved 100% branch coverage in August of this year. Clearly a high watermark for them but also a goal that takes a very long time to realise.

CNR

It could so easily have used cloning as a theme to arrive at “could not replicate” instead

I’m aware it’s been some time since I posted. However, in lieu of a “proper” blog post, I can tell you I’ve started reading Jerry Weinberg’s Perfect Software and will likely discuss it in a later post. I’ve heard good things about Jerry so it should be an informative read…

One of the side effects of testing is you develop a distrust of most software products, especially web sites. It’s a bit like the critic who can never really enjoy a show because he can never snap out of “review mode”. So it’s with some trepidation that I ever enter any punctuation marks into search fields, especially that most-abused of marks the apostrophe. Combine that with the fact that conscientious developers should always escape/strip quotation marks when passing user input to a query and you get this:

http://www.myvouchercodes.co.uk/code-store-search/domino’s

TheTrainLine.com deserves praise as it will let you check if two single fares are cheaper than the equivalent return fare when you search for a journey. They almost always are and this fantastic feature saves you a packet for just about every return trip you make. (Such are the vagaries of the ATOC/National Rail fares system, but there we are.)

Anyway I wanted to remind myself what time my train goes tonight, so checked the booking/journey confirmation email. However it has a bit of a flaw in presenting a critical piece of information (which I have highlighted):

Dear MR PAUL BERRY,

With your journey approaching we wanted to remind you of your travel details.

Collection Reference: 352CHH7W

Be Sure to bring this reference with you along with the payment card used to make the booking as both will be required to collect your ticket from the Self Service Ticket Machine.

From: SHEFFIELD

To: LONDON ST PANCRAS INTL

Date: 16/01/2009

Departure time: 0.76875

All your journey details are available in the my account section of our website.

If you need help with your booking, please visit our help section or call us on 0871 244 1545*

We hope you have a safe and enjoyable journey,

thetrainline team

*Please note that calls to this number cost 10p per minute plus network extras

Now this looks like a classic “time stored as decimal” formatting issue, as you sometimes see on raw spreadsheet data. The mystery is solved if you consider the time as a fraction of a day. 24 hrs * 0.76875 = 18.45 hrs or 18:27 in sexagesimal. Double-checking the times (since I’d already lost trust in the accuracy of the data) on National Rail Enquiries does indeed show a train departing Sheffield for London at 18:27 tonight.

I’m just thinking a bit further here of the person who goes through the trouble of doing the maths to work out the time but doesn’t do the final step so they get 18.45, which looks reasonable enough, and turn up for their quarter-to-seven train having missed it by 15 or so minutes! (Of course if the decimal hours come out at, say, 18.67 then that’s clearly not a time and in itself would be a prompt to complete the conversion but the above scenario still stands.)

What was Steve Krug‘s mantra again?

I wonder how many phone calls they get because of this? Could be a nice little earner! 😉

Sometimes you just grind to a halt when testing: you’ve exhausted the test plan, tried most reasonable combinations of inputs, even thought of and exploited the most devious set of circumstances to trip up the application, all that stuff. Now your eyes are starting to glaze over. It happens, especially if there are no opportunities to pair test or hand over to a colleague.

Well the following technique worked for me and I didn’t even know that it was anything special or realise it would yield a fresh batch of bugs. So here’s what you do…

List stuff!

Yep, look at the whole screen and list things in the order they appear. I mean actually type the headings, titles, etc, into a spreadsheet or whatever your favourite note-taking method is. Then compare these views with the functional and technical specs, if available. Your mileage may vary, but you might find some subtle bugs.

It’s really that simple. And it can work: after a while you become blind to the design, layout and behaviour of the software under test. Force yourself to read everything. You may feel you’re going over things you’ve already seen a million times but truth is you may never have looked at them that closely to begin with.

As an example, I was testing an application which stored the names, addresses, etc of a list of clients the customer dealt with regularly. The list was rendered as a datasheet: column headings across the top, draggable, clickable to sort up/down. Nothing special at all. Then I noticed one of the columns was called “Country”. Did the customer have any international clients? Was there a field on the input form to specify a country? In any case, were these even requirements? No to all three. So there was a completely useless column sat there (copy-and-paste code?) and neither I nor anyone else had noticed after testing for over a month. There’s the rub: after seeing the same old screens day in day out I had become as good as blind to them, until I adopted a fresh way of looking at things…

What’s wrong with just stating “when prompted, cancel request” in your test case? Well, just think about the ways in which a dialogue box can be dismissed:

* click the close box (if present)
* click the [Cancel] button
* press escape
* press tab until the [Cancel] button is highlighted then press space
* press tab until the [Cancel] button is highlighted then press enter
* press tab until the [Cancel] button is highlighted then press return
* press alt-c or whatever letter in “Cancel” is underlined when alt is held down (if present)

Do you test each of the seven ways with each dialogue box you see? Why not? Because they all do the same thing? Well, one of the methods might trigger something the others do not (and yes, it has happened to me). Then again, even if I did adhere to these strict methods there might be yet another legitimate way of hitting [Cancel] that I’ve never thought of.

But a man can become paranoid. Clearly you have to be realistic in how you test and that will only come with experience. Don’t sweat the small stuff and don’t get bogged down in detail. Let’s take another look at these seven methods of invoking [Cancel]. They can be split into two groups: those triggered by mouse action and those triggered by the keyboard. Suddenly not so daunting.