User Experience Research

Lessons I’ve learned developing, integrating & managing a user experience research program in a B2B digital technology company

“Good timber does not grow with ease” — Malloch

In my experience, product and executive teams want to push for user-driven designs and business decisions, however, in practice, it is challenging to implement. In a 2017 User Testing report, “81% of the executives surveyed said user research helps make their company more efficient. A further 86% said that user research improves the condition of their products and services.”

In this article, I discuss three topics that have the potential (when approached incorrectly) to undermine your efforts integrating user experience research into an existing design-process. These topics are:

  1. Research-related disruption
  2. Resistance to change &
  3. The importance of resourcefulness

1. Expect to bring about disruption.

Yes, you might enter an existing work process not quite ready for a user experience researcher.

You might welcome a user experience designer to collaborate with you regarding their task-related concerns. As a result of the collaboration, you might conclude a unique study is required. The proposed research might uncover actionable insights, which the designer might implement to better their design deliverable. In such a case, you have intentionally introduced research between the user experience designer and the user interface designer (the person who is meant to be next in line in the work process). You have disrupted a familiar work process in the hopes of ensuring quality deliverables. Given this disruption, how do you best navigate it?

Process & potential solutions.

The mindset. In his talk, entitled, “People, Products and Jetlag: Creativity Through Empathy” Google’s Jens Riegelsberger discusses the importance of empathy in user research (see here). He states, “Measure what counts — not just what you can count.” The spirit of this particular statement resonated with me and is central to how I approach research. Prior to proposing research, I’ve learned to run empathy-driven collaborative sessions. I’ve experienced three positive outcomes as a result of taking a collaborative approach.

  • First, you will accurately measure what counts, and in turn, will offer high-quality actionable insights to your team.
  • Second, when disruption occurs (as a result of research) your team will be more likely to welcome it.
  • Lastly, as a result of actively involving team members in empathy-driven collaborations, there is significantly less resistance from their part (than there would be otherwise).

The know-how. A prerequisite to a successful integration of a research process (into an existing design process) is direct experience with all research steps. In general, there are three phases involved in research (diagram below):

  1. Planning
  2. Execution &
  3. Reporting

In the planning stage, you will have to consider the following five key steps:

  1. Have we defined the problem statement?
  2. Have we conducted a needs analysis?
  3. Have we examined historical data?
  4. Have we stated a priori expectations?
  5. Have we agreed upon a research design?

During execution, you will have to consider the following four key steps:

  1. Have we completed administrative tasks (e.g., non-disclosure documents)?
  2. Have we agreed upon recruitment strategies?
  3. Have we completed data collection?
  4. Have we completed data analysis?

In the last phase, reporting, you will have to consider the following five key steps:

  1. Have we identified actionable insights?
  2. Have we completed (and disclosed) an internal research report?
  3. Have we completed (and disclosed) an external research report?
  4. Have we ensured transparency?
  5. Have we ensured confidential archival?

Knowing these fourteen steps will allow you to be strategic in your integration efforts.

Personal note. I drew upon my research experience to help time integration steps. At times, I explicitly requested and implemented change (e.g., the necessary development of a participant pool). At other times, I proposed research-forward ideas and deliberately stepped back from further action, in order to allow ideas to percolate gradually among my team.

2. Expect a bit of resistance.

Yes, you might experience initial resistance.

Interestingly, there are designers who feel hesitant in permitting researchers to test their designs.

Process & potential solutions.

Empathy. On multiple occasions, I stepped-into-the-shoes of hesitant designers, and through a first-person vantage point, I attempted to breathe in their diverse world-views. This practice helped to shape my future interactions. In speaking with a hesitant user interface designer, I made sure to explicitly express my aim is to test designs — not him. I attempted to reassure him by stating outcomes of research are not reflections of his work performance. Further, I made sure to time my supportive language before data-collection and report-disclosure (which I have learned, are high-emotion moments for a team).

Language. To further ease the transition for the team, I simplified my language to ensure team comprehension and efficiency. For example, instead of using the word affect use emotion, instead of salience use obvious and instead of latent use underlying. Overall, distill information and use concise language whenever possible.

Consistency trumps intensity. Consistent effort is an essential ingredient to collective UX research acceptance and adoption. Intense effort, exerted at one-point-in-time (perhaps in the form of a group workshop), will not result in the long-term integration changes you are perhaps hoping for.

You might be thinking: “But I just don’t get it — why would a designer be hesitant in letting someone else test their designs?”

One potential answer to this question requires an understanding of the educational background of designers. I’ve learned that designers are formally taught to “own” their work. Perhaps, as a result, designers are more likely to construe their creative expressions as reflections of their self-concept (i.e., “My art. My designs.”). Researchers, especially empirical researchers, on the other hand, are formally taught to divorce themselves from the fruits of their labor. This teaching helps to ensure (as much as possible) a central tenet of empiricism, namely, objectivity. Without a firm hold on objectivity, researchers are at risk of contaminating data-sets and misconstruing research findings to favor a particular outcome.

Personal note. I drew upon my teaching experiences to ease the integration process by teaching in bite-size amounts whenever necessary.

3. Expect to be resourceful.

Yes, you should not expect that all research-related resources are ready for you upon your arrival.

In a B2C environment, you might have, for example, direct access to your end-users. Apple, for example, has multiple and diverse pools of participants ready to partake in user experience research (to investigate, perhaps, a new accessibility feature of the latest iPhone). Participant pools usually consist of real end-users but some pools might be composed of potential users. User experience researchers, in these B2C environments, as you might correctly conclude, would utilize existing participant pools for research purposes many times throughout the year.

Conducting user experience research in a B2B environment, however, poses unique challenges not readily seen in a B2C environment. As a B2B user experience researcher, you might still be expected to investigate users, but they might not be your company’s clientèle, instead, your user group of interest might be your client’s users. Without direct access to these users you might have to formally request access from your client in order to conduct research. As you might imagine, such administrative processes are time consuming. If direct access is not possible, you might have to assemble participant samples that are as representative as possible of the end-user group.

Process & potential solutions.

The mindset. You might have come across the following quote, “Success is not about your resources. It’s about how resourceful you are with what you have.” (Tony Robbins). Adopting a mindset oriented in resourcefulness is essential to research success.

Take productive action. I made the choice to set up a meeting with upper management. I disclosed that to ensure quality research is delivered, I would have to develop and curate a participant pool for research purposes. Once I received the green-light, I began the logistics involved in mass-advertising, participant-screening, participant-recruitment, documentation (i.e., information-consent & non-disclosure forms) and expense-tracking (e.g., participant-remuneration). My immediate aim was to develop a large and representative sample for the ultimate aim of delivering the highest-quality research.

Expect to take initiative during times of ambiguity. Do not wait to be told to initiate action about subject matters which fall under your area of expertise. Yes, it is true that ambiguity engenders discomfort. I’ve personally found discomfort to be central in both personal and professional growth.

A note to you. I hope you might transform my insights to increase your pool of knowledge. In turn, perhaps, you might better self-forecast challenges and navigate with greater ease in the ever-changing field of research.Tomorrow, when I learn something new, my take on a particular topic might change. You might have a different outlook on one or numerous points I’ve described here. I welcome diversity of thought and would love to learn your perspective (feel free to say hello: h2alhome@uwaterloo.ca).

A visual of the research process:

Thanks to Alex Millar for reading a draft of this article.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store