By Richard Whitehand, March 2002
It is becoming more common to hear of IT development projects involving users in their development process. Good news, I hear you cry! Unfortunately, the extent and method of end-user involvement varies greatly and some project managers have become disillusioned as a result.
Infact, on the surface its quite possible for a development project to look like it has done "all the right things" and involved users at key stages in the process, but still completely fail to produce a usable product at the end. How is this possible?!
Described below are some user-centred activities that an organisation* (footnote) we recently talked with carried out to support the development of their new B2B (business-to-business) e-commerce web site. The activities themselves appear sensible enough, but underneath is a description of what the activities actually involved and their consequences.
The organisation conducted a series of interviews and workshops together with some of its most important clients. The purpose of the activity was to understand the needs of potential users in terms of what functions should be present on the web site, and how they would be used.
To reach users the developers contacted the marketing department, who recruited people using their contacts at key customer organisations. These 'users' turned out to be key purcashing decision makers, and whilst able to provide useful input, were not doing the day-to-day order processing of the actual end-users. Some participants were familiar with the work of their purcashing staff, so a few useful insights were gained, but unfortunately a lot of information was missed.
As the most important clients had been invited to participate, the needs of 'less important' customers were not considered. True, the 'top 10' clients were responsible for 30-40% of turnover, however the other customers combined were responsible for more and unfortunately they had different needs. Two critical differences were:
To ensure that designs met user needs, the development organisation asked several of the participants in the requirements gathering activity to participate regularly in design meetings. The users saw and commented on the designs on an almost weekly basis, and participated in development discussions with designers and programmers.
Due to the extensive and repetitive nature of their participation, the users were not only thinking more like designers and developers than actual end-users, but were also much more familiar with how the design really worked than 'normal' end-users would be. Their comments and feedback were given great weight when making design decisions and resulted in an overly complex and 'feature-rich' user interface.
To further ensure that design work stayed 'on track' and that the user interface was easy to use, the organisation tested the interface with users.
For this activity the organisation held a series of focus groups, where the design was demonstrated to users and they were then asked for feedback. As the person running the focus group knew how to use the interface and pressed all the correct buttons in the correct order then users thought the interface looked fairly easy to use. They made comments on the colour scheme and some of the wording in the interface, but otherwise felt that everything was okay.
Unfortunately in the 'real world' end-users don't have somebody standing next to them to demonstrate the interface and must be able to interact effectively with the interface themselves, carrying out real purcashing tasks. The 'group test' approach almost entirely missed this point and thus resulted in developers believing that, with a little 'polishing', the user interface would work well.
A conservative estimate indicated that 4 of 5 users (80%) were unable to find the products they wanted and/or complete an order using the site 'as launched'. Whilst not representing completely 'lost' customers (there was still a paper catalogue and telephone sales support) this cannot be regarded as a success story in user involvement!
Properly planned and executed user involvement can make huge improvements in the success of an e-commerce web site. Involving the wrong users in the wrong way is a receipe for disaster.
* Footnote:
This article was written after telephone conversations and personal meetings with developers and project managers for several large Swedish commercial organisations. One of these forms the basis for the article. It should be noted that some details of user involvement in activities were not known (or not divulged) by the person we spoke with at this particular organisation and are thus based on typical findings from others.
You might like to read some of our other editorials.
Please send us an email if you have any comments or suggestions!