Tuesday, 11 June 2013

User research in an agile environment

I was invited to speak at the Global Reviews Digital Leader Summit last week which focused on driving customer engagement, sales and program velocity. The summit was a tremendous success with an impressive list of attendees and speakers.  It was daunting to speak after Chris Ho from NAB and Barry Newstead from Australia Post, who provided insightful and engaging presentations. That said, I enjoyed presenting and being able to share my views on the importance of user research in web design within an agile development process.

I thought I would take the opportunity to share some of my presentation here. 

The increased adoption of the agile development process within the digital industry presents a challenge for incorporating rigorous research as part of the design process, particularly in the way that we have traditionally approached research. This is due to the time involved in doing research with rigour.

Not including research can result in building products which do not meet user needs. This is ironic because this is also one of the reasons for organisations embracing agile. That is to ensure the end product aligns with user requirements instead of spending years building a product which is not what people want.

At its essence research is about extracting information. However that information only has value if it is accurate and can be relied upon. To make doing research worthwhile we need to do it properly.

The problem: Conducting rigorous research within an agile development process

Research can be a slow process, potentially incompatible with the rapid iterations of Agile. This is due to:
  • Recruiting participants for research activities, for qualitative activities like interviews or moderated usability testing, which can take as much as 2-3 weeks
  • Conducting qualitative activities such as interviews or contextual enquiry is time intensive. This typically constitutes days or weeks of work as opposed to hours.
  • The analysis phase can also be time intensive; identifying the insights from research requires time opposed to regurgitating observations and direct feedback from users.

Incorporating design research into an agile process

Much like the culture shift which is required for going ‘agile’, research also needs to become a part of the culture. Here are some ways to include research in agile.

1.    Effective planning: Research activities must be planned for. Sprint zero can be used to define the research needs generally, as well as the information sought from research. This could include any outstanding questions regarding users and identifying the design assumptions which require validation. Research activities can then be scheduled for upcoming sprints which can accommodate the time involved with recruitment.  For example we can schedule a round of usability testing for in 2 weeks time, to test the primary design assumptions and include any questions which arise over coming weeks.

2.       Using time efficient techniques and tools: Online quantitative techniques such as un-moderated usability testing, tree jack studies for testing IA, surveys and online card sorting can be conducted without the lengthy fieldwork periods associated with qual techniques. 

Some of the great tools out there include Userzoom which is a comprehensive suite of UX research methods including un-moderated online usability testing, cards sorting, IA testing and survey capability. It is an enterprise level tool which is used by Global Reviews. Optimal workshop, is another example of the tools available, offering card sorting and IA testing.

A further factor reducing the time associated with utilising these tools, is their great analysis functionality which can dramatically reduce the amount of time taken to complete analysis in comparison to in-person methods. Time is also saved in the collection of data.

3.       Placing the right systems and processes in place: A key requirement is setting up access to customers to be able to get rapid feedback. A database of customers who are willing to participate in research is ideal. This can be effectively supported by an active social media presence. By having a system to get access to customers quickly it is possible rapidly reduce the time involved in recruitment.

Customer can be asked to get involved in research during a sign-up process or via communication channels such as email.

Another approach is to schedule research which takes place at set intervals. For example scheduling customer interviews once a month, every month, regardless of the research needs and information required. I heard a great interview with Tomer Sharon on Gerry Gaffney’s fantastic UX Pod  where he talked about using this type of approach at Google. He conducts ‘Fieldwork Fridays’ where he gets software engineers to conduct the research with customers on a regular basis and argues that this has a huge positive impact on their products.

Both of these approaches overcome the shortcomings of long lead times for recruitment.
The key here is to have customers ready to participate in research activities at short notice.

Example: bringing it together

To provide an example, I recently conducted a card-sort within an agile team. There were questions about how the IA should be labelled and grouped; an understanding of how customers thought about the content was core to creating a successful design. Evidence was also required to justify decisions to stakeholders. On a Wednesday morning I created the card sort in Optimal Sort with around 40 cards. At midday it was sent out to customers. By 10AM the next day I was analysing the results and in the afternoon I was able to provide feedback the product manager and the rest of the team. This was a great example of having the right systems in place and making use of the right tool to provide rigorous and highly rapid feedback.