Postscript: More birth pangs of government 2.0

In a long post on his blog titled 'Usability myths and professionals' Alastair Campbell (no, not that one, the other one) of Nomensa has responded to comments I made about their advice to government web workers (they're at the end of my original long post).

Says Andrew:

A recent post by a local authority web officer was fairly frustrating for me, as it perpetuates several myths in usability, as well as calling into question my motives.

That's right. I questioned that every help route seemed to end up back at a professional, aka 'pay-me'! Andrew's response only confirmed this for me! More about the 'myths' follows.

Nomensa were contracted in 2006 by the DCLG (Department for Communities + Local Government), which ran the government portal DirectGov and the centralised LocalDirectgov programme Someone in Whitehall had finally realised that there was no usability advice so - for a while - Nomensa provided CDs, online usability guidance and a helpdesk, which was used but not that much.

Andrew notes that eGov critic, the publication Public Sector Forums, had "almost nothing but good words" to say about their work at the time. But he then thinks because I've posted on their forum that I'm a 'member', so he's surprised that I'm "now taking umbrage". Er, I don't represent PSF, neither do the other hundreds of 'members'. I disagree with them quite often, actually.

For the record, I'm not taking umbrage at the advice in total. Having it was useful, but it was late. Looking at it again now I'm sure it would need changes. It did seem extremely targeted - so not always useful, not obviously - but there were good reasons for that. All of which are now irrelevant because it's been moved, relinked to and parked - not added to or promoted. This appears to be it on usability advice/leadership from Whitehall.

Andrew claims authorship of a quote which I used, actually from the guidance FAQ:

No usability guideline is black and white, and the context and users have to be taken into consideration.

This is what I said about that:

Whoever wrote this has a vested interest, pushing their expertise— are they really saying that someone like Jakob Nielsen doesn’t make basic, apply to all, guidance? That ordinary web workers have nothing to learn from Nielsen or any of the others in my links list? That only filtered and packaged government-approved usability guidance is kosher?

Noting that they did have links in their guidance (after prodding to useit.com, from my recollection), Andrew:

No, not where people are involved.
Jakob Nielsen has done much to publicize usability, but you do have to take care when things are simplified too much, or assumed to be sacred. For example, he used to say people wouldn’t scroll (mistake 6), but this isn’t the case anymore (e.g. 22% scroll to the bottom in this sample, and most scrolled to some degree).

"Much" — I can't think of anyone bar maybe Steve Krug who's done more? "Used to" — exactly, he's changed his advice over the years in some respects. Most recently and most famously on providing the actual (negative) results on banner ads. But on scrolling with a very quick Google I find him saying in 1997:

Scrolling Now Allowed
In early studies, I found that only 10% of Web users would scroll a navigation page to see any links that were not visible in the initial display. The vast majority of users would make their selection from those links they could see without scrolling. In retrospect, I believe this was due to people treating a set of Web options like they would treat a dialog box: You always design dialog boxes so that all choices are visible ...

In 2002:

Users hate scrolling left to right. Vertical scrolling seems to be okay, maybe because it's much more common.

It's obvious that scrolling behaviour has changed. Driven, I think, largely by Google. Obviously Jakob's advice has changed as he's observed changing behaviour.

What I think Jakob does especially well, particularly when he's being an evangelist, is remind people that for many if not most users, the Web is hard. Lots and lots of users do fail tasks all the time. i.e. not everybody scrolls. Loudly saying this, reminding web people, makes him a curmudgeon for many (you fellow professionals have your own issues with him I guess).

In any case you are dealing with percentages, statistics, and optimising. Not clear guidelines that work for all, which is what I was trying to suggest.

What about heuristics?! I know from fieldwork that starting from basic heuristic points works with giving people basic rules. 'Help users recognize, diagnose, and recover from errors' is guidance that's always there, or should be.

Any usability finding has to be in the context of who, when and what. It’s actually in the definition of usability (emphasis mine):
the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use.

Of course, but that's not what Discount Methods are about. They are about a way that you can spot woods from trees in a very particular context.

Nielsen just posted about how they'd spent gawd knows how much testing a US Census website. They wanted people to find the US Population number. Despite it staring them in the face on the homepage, simply because it was in red, no one saw it. Literally no one.

Do you not think a quick whizz round the vicinity of the US Census HQ using discount methods might have saved them some money for the same result?

I had a very similar experience recently with a website, actually with an element - a very important element - in that very position. If we hadn't discount tested, we might have missed it.

Discount testing does its straightforward job. A very important job that is transformative in a way that just following guidance can't match. That's all.

Andrew:

Many sites would benefit from quick internal usability testing at various stages of the process, that is only to be encouraged. But you do run the risk of finding out what you want to hear, or using the wrong tool for the job. Again, it depends. If people are asking for general guidelines to use, it’s a good indicator that help is needed with the methodology.

And that's a bad thing? The only way to try to avoid "finding out what you want to hear, or using the wrong tool for the job" is by following some advice on how to do it and to externally test it. If you're after woods/trees and not trying to do much more. You seem to suggest that finding common errors either isn't consistently possible or is bound to be heavily discounted by bad methods. How do you know this?

What I would say is that personality and things like experience of dealing with the public for testers are key areas to nail down in such advice. I'm reluctant to provide scripts but I can see how that would help.

This is an area from my experience that needs work. particularly because there's one huge benefit from discount testing - meeting the general public, the customers. (And another - answering internal forces, such as those who propose unusable web elements.)

In a presentation I gave last year I have a section about when you need a professional (I suggest, for one, at the very beginning of the process. The real aim should be that any final testing confirms and adds minor tweaks. Unfortunately, as you'd know, this is when many people start).

Fact is, only the biggest, wealthiest websites can afford much use of professionals. So people need to know when they need them, what to prioritise. They need methods to test, not just run against guidance, for all the rest of the time, when there isn't a budget for professionals (or when they've devoted a chunk to accessibility).

Simply spotting woods from trees would do everyone enormous good and - as you agree - should be encouraged. But how? You're not doing it, are you? Who is apart from Jakob? You do appear at least to have a vested interest in not doing this.

Anyone can claim to be a usability expert, just like anyone can set up a web site. But like web development, there is a need for professionals.

I don't claim to be a usability expert, never have. But you come across that your advice is that you always need professionals. That any ideas you - lowly web worker, developer, whatever - might have about usability are ridiculous, naff, prone to error and - well - just forget it. And that's - natch - disempowering, doesn't advance the cause of usability and ultimately doesn't benefit customers.

So. to be positive, why don't you write discount testing advice (there was none in the LocalDirectgov work)? Plus better advice on when you need a professional and what you can do yourself? You are the expert after all.

• Postscript: discussion continues on Alasdair's blog.