Research Lab vs. Wild: which do you prefer?

Are you a UX designer or researcher who has the luxury to do some research in a lab? Not every organization has an on-site facility where things like empathy interviews and usability tests can take place. But for those that do, let’s explore the pros and cons of the lab and working in the wild.

I have had the ability to work in both a professional usability lab and perform ethnographic research in a person’s natural environment. I have my opinions about both, but I will reserve those until the end. First, let’s cover a bit about the difference between working with users in these environments.

Working in a Lab

A lab is a great set up. The tables and chairs are all set up for you. It’s quiet and protected from outside noises a person will usually encounter in a public place. The lab can be technically fitted with luxuries like video and audio recording capabilities. They usually have power sources, so users can use in-house and already loaded equipment like computers and mobile devices. When an organization has a lab, they often have a budget for fancy recording devices like a sled for recording mobile actions or eye tracking software. A lab is a great place to get controlled, scripted and qualified studies and act as a calm testing environment.

But a lab is not real. It’s not a person’s real home or office. It’s not the actual place a person would be performing an action like shopping online or texting with friends. A lab is a fake set up requiring a person to suspend belief and pretend they are in their natural environment. And this can be done to a point. But if people surf the web with their feet up on the table, with a tablet on their lap and a mobile device in their hand, using pre-described lab equipment will not realistically record a person’s natural behavior. Plus, just being put in a lab setting puts a person on edge and makes them feel uncomfortable. We all feel strange when we know we are being recorded. We probably even act differently and are not even aware of it. Yes, knowing these factors, we as researchers should try to combat these fears and make the environment feel as homey as possible. This just cannot be completely done since it truly is a false environment as far as the participant is concerned.

Despite a lab having many technical and environmental luxuries, you can see why many people opt to view people in their natural environment. Not to mention that most companies cannot afford to have an on site lab or pay to use external lab facilities. So for those who have no lab, it’s time to talk about jumping in to the wild and observing people in their natural habitat

Working in the wild

Observing participants in their natural environment provides many advantages. First, it is cost prohibitive. We won’t have to pay for lab facilities to see a person work in their home or office. Second, people feel more comfortable if they are on their own turf. It’s their home, so they know where to sit, how they work and where they normally do things. Also, the beauty of performing a contextual inquiry, or observing someone in their natural environment, is that you can take note of the items around them. How do they sit? What is their set up? What items do they have around them? What items do they use while interacting online? What is around them that is distracting? Who are they interacting with? And so much more. When you are in a person’s place, you can see the special notes and shortcuts they keep handy. Do they have their password written on a Post It note right on the computer monitor? All of these little environmental clues are golden nuggets that cannot be copied in a lab environment. Finally, one more benefit of people working in their own space is that you are getting a much more accurate picture of who they are and how they fit in to their space. You can never gain this kind of insight when they are sitting in a foreign lab using foreign equipment and getting instruction on how they can do things rather than exploring an interface on their own.

Despite the many advantages of entering a person’s real world and viewing how they really work in this context, there are also issues in working within these surroundings. Going to someone’s home many not be the safest situation because you don’t know exactly what you are getting in to. Make sure you travel with at least one “note taking” buddy to ensure your safety. Also you are very likely to encounter obstacles like noise, distractions, space limitations, and inaccessibility that can make researching in an environment tricky at best. Plus, when observing people in their surroundings, they can fall victim to The Hawthorne Affect. Basically, that is when people modify their normal behavior when they are being observed. Just keep in mind that this could happen with any, if not all participants so plan accordingly. Keep these factors in mind, and cater your research to accommodate such potential pitfalls. However, don’t let these deter you from going in to the wild. From my experience, I have gathered much richer insight by seeing a person reacting to their environment in real time, as opposed to the comfortable, staged lab setting.

What do I choose? Lab or Environment?

As is the classic answer to so many questions. It depends…. Obviously if you don’t have a lab, then that decision is easy: go out in to the wild. But if you do have a lab, I would encourage you to still go out of the building and see people in their natural space. It clearly provides a context that you will never get in a lab.

Rapid Prototyping with Tom Chi

Jennifer Blatz UX Design presentation
Sharing with your team when they cannot participate in an event or workshop is a good skill to have in your UX Designer toolbox.

Our office recently had the honor of hosting renowned GoogleX prototyper Tom Chi. He came in to our office to work with Product Managers on learning the value of testing fast and testing now! I had a chance to participate in the session as both a user, working through prototypes, and as part of a team building the prototypes for testing.

I have to admit a lot of what Tom covered was not completely new:

  • Find the quickest path to the experience
  • Test early and test often
  • Don’t guess. Learn
  • Don’t “fail.” Learn
  • Stop talking and start doing
  • Get in front of your users and get their feedback

But there were a couple of concepts that really resonated with me and thought they provided value to the session.

Drive conjectures to experiments. Experiments drive decisions.

Conjectures are the same thing as guesses. In other words, people tend to get stuck in “analysis paralysis” and over talking about the situation. In fact, a lot of these discussions are not reality based and is a process of throwing out personals opinions. It might be driven by the best intentions. But these conversations often go on for too long and are never backed up by actual user research. So encourage your group to stop talking and start doing.

The way we did this in the Tom Chi Prototyping session was to stop talking and we each sketched ideas quietly for 3 minutes. They key here is to sketch individually in silence. There should be no talking during the sketching exercise so that each person is exploring individual ideas without the influence from others. After sketching, the ideas are then shared with the group. One or more idea is selected as the “champion.” And this this rough sketch is what should be tested with users. No need to create a higher fidelity version of the sketch. Just show them the rough sketch and get quick feedback before you are too emotionally and technically invested.

Focus on people’s energy

Whenever there is energy, that means something that matters is happening. This energy can be positive or negative. When a customer gets exciting about something, pay attention to that. And the same is true when they show angry excitement as well. It’s these magic “energy” moments that really improve or ruin an experience. So tweak those energy points to make them awesome. One particular example Tom Chi mentioned was Uber. Most of the Uber experience is just like riding in a cab. It’s the few seconds that are different in that experience is what matters.

Don’t lead the witness

Finally, I want to talk about one lesson I learned by going through these sessions. Most of the people participating in our sessions were not researchers. In fact, there were product managers and designers. These are empathetic people who are excited about getting great feedback from users. But they don’t understand that the way they ask questions can skew the response from the participant. It is better to ask broad, open-ended questions rather than helping the participants by giving them examples. It is these tactics that researchers know how to do, but others might not know. They don’t understand that by providing examples for the person to think about, they might be blocking other examples they could possibly come up with on their own. With a little coaching, I know that product managers, designers and other non researchers can learn effective ways to ask non-leading questions.

Take away

I thought the most valuable aspect of bringing Tom Chi in house was to empower non-designers. I think he gave everyone confidence that they can explore ideas, sketch the ideas, and get quick feedback from customers. I hope this process takes off and continues well in to the future.

To read more of my notes, read the Tom Chi Prototyping Workshop presentation that I shared with my team.