- User research - different approaches and methods
User research - different approaches and methods
This is the second part of my fourth lecture at HITLab, Canterbury University, Christchurch, New Zealand about user research. I am presenting the three levels of understanding user needs and the methods that correspond with investigating these needs. The idea is to show how different methods enable a designer to dig for different insights and how to conduct exemplary studies for each type of the method.
- user research: dierent approaches and methods aga szstek(at)gmail.com
- what people level of knowledge research methods Fraukje Sleesvijk Visser
- what people level of knowledge tell think explicit interviews research methods Fraukje Sleesvijk Visser
- what people level of knowledge tell think do act explicit observational interviews observations research methods Fraukje Sleesvijk Visser
- what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods Fraukje Sleesvijk Visser
- before discussing the dierent methods it is crucial to know about the concurrent and retrospective experiences
- - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed
- - when people report on their current feelings the feelings themselves are accessible to introspection allowing for accurate reports - aective experiences are eeting and become not available once the feeling is gone - once the feeling disappears the aective experiences need to be reconstructed - reports of past feelings are based on semantic knowledge - when asked how theyusuallyfeel during a particular activity, people draw on their general beliefs about that activity - the actual experience does not gure prominently in these reports because the experience itself is no longer accessible - the same knowledge is a basis for predicting future feelings, for which episodic information is not available to begin with
- so, lets discuss the methods
- what people level of knowledge tell think explicit interviews research methods
- introspective versus retrospective methods
- introspective methods Analysis of present behaviour and gathering of preferences by combining observations and interviews in context: - cognitive walkthorugh: heuristic identication of problems connected to the current ways of behaviour and interaction - think aloud: heuristic identication of problems arising from the current state of interaction through verbal articulation of user observations while interacting with the solution - co-discovery: a method where two users collaborate in order to resolve the problems found in the process of interaction with the current solution - Contextual Inquiry: gathering of detailed information about behaviour and motivations of users while the user is actually performing a given task
- retrospective methods Support for retrospective assessment of user preferences towards a given solution: - interviews: a discussion with the user aiming to collect rich material about his / her interactions and attitudes - focus groups: group discussion about a given problem or solution - retrospective analysis of behaviour: and interview supported by a qualitative recoding of user behaviour (e.g., video or audio recordings)
- when to apply? - exploration phase - to get a better understanding - to nd inspiration for further research - requirements validation - to check whether study assumptions are correct - evaluation - to discuss and verify design decisions
- types of studies
- - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions
- - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable
- - fully structured: the entire study takes place according to the earlier prepared scenario - good for comparison across individuals - more extensive answers comparing to written surveys - inability ask unplanned questions - semi-structured: there are certain guidelines to the elements of the study but the order is unimportant - pre-specied questions - questions as a starting point for discussion - digression from the script is acceptable - unstructured: going with the ow, only the initial question and a possible list of topics is dened - good for open ended exploration - dicult to conduct and analyse
- study questions - close-ended - simple questions get simple answers - On a scale of 1-10, 10 being best, how did you like the prototype? - Did you like the prototype? - easy to analyse, but may not be that informative - open-ended - invite elaboration and discussion - build conversation and stories - What did you think about the prototype? - avoid - long questions, jargon, leading questions, unconscious biases, negative questions
- - ask why? - even when you think you know the answer, ask people why they do or say things - encourage stories - whether or not the stories people tell are true, they reveal how they think about the world - look for inconsistencies - sometimes what people say and what they do are dierent. These inconsistencies often hide interesting insights - observe nonverbal cues - be aware of body language and emotions. - do not be afraid of silence - if you allow for silence, a person can reect on what theyve just said and may reveal something deeper
- preparing the study - brainstorm questions - identify and order themes - pilot the study - write an interview guide - prepare for capturing the interview - prepare logistic backup
- conducting the study - you are the host - be friendly, respectful and non-judgmental - do not bias through body language - be exible (unless fully structured) but keep things on track - ask for clarications if something is unclear - listen (dont talk too much) and read between the lines - be consistent - outline - briey introduce testing goals - complete paperwork (informed consent) - simple questions rst, hard questions later
- capturing the study - have someone to help taking notes or do the audio or video recording - capture insights, non-verbal responses, etc. - balance between detail and conciseness - summarize immediately after the interview - do transcriptions - respect privacy and anonymity
- debrieng - ask for any nal comments - provide more detail about research goals - oer a brief summary of ndings - turn o recording devices - saythanks! - reect and summarize notes immediately
- what people level of knowledge tell think do act explicit observational interviews observations research methods
- - roots in ethnographic research - observing people in action - recording what is being observed - helping the researcher learn perspectives held by participants - often used in conjunction with other methods (e.g. interviews, focus groups, content analysis)
- - observations: observing and recording of user behaviour without a direct contact between the user and the researcher - shadowing: direct observation of user behaviour by the researcher without intervening with his / her actions - interaction logging: logging previously dened behaviours and system states that pertain to the specied user behaviour - diaries: systematic reporting by the user his / her interactions in a form of a diary or a blog - experience sampling: a systematic way of having participants provide samples of their ongoing behavior when participants' reports are dependent on either a signal, pre-established intervals or the occurrence of some event - day reconstruction method: capturing daily experiences of the users at the end of each day where participants are asked to listed all activities of the day that somehow related to the product or a situation that is under investigation
- data collection - eld notes - video - audio - transcripts - behaviour matrix - situational drawings
- pros and cons - exible and open-ended - directly measures behavior - gives the researcher insider view - allows for morphing of the study - hard to be an insider - invasive and intrusive - dicult to record - subjective - Hawthorne eect - time consuming - not generalizable - does not measure cognitive or aective aspects
- phases of observational research - phase 1: develop positive relationships with participants, gatekeeper, etc. - phase 2: improve design, refocus, redene questions after initial observations (may be ongoing) - phase 3: select additional participants as necessary - phase 4: follow up and probe deeper
- ethical considerations - how much do you disclose? - make your intentions clear - get an informed consent - develop a code of practice - maintain condentiality
- (Mack, et al., 2005)
- being an eective observer - know what you are researching - check in with your lens, biases, experiences and expectations -- have a blank mind - go where people are engaged in their daily lives and nd the right informants - understand the culture - rehearse how you will explain the purpose of your study - decide how you will do the observation and collect the notes - document what you observe without expectations - summarize and expand on eld notes as soon as possible - be open to the research problem reformulation
- what people level of knowledge tell think do act know feel dream explicit observational subconscious latent deep interviews observations generative techniques research methods
- using design as a research tool
- cultural probes
- - tools for better understanding of peoples latent needs and dreams - small craft objects used in empathic engagements with individuals around issues centered on self- identity and personal signicance - their form should reect the research question and its context - asking questions by applying a delicate, provoking and creative means - and oering intriguing ways to consider the asked questions and to look for surprising answers
- understanding what people do with their invoices AgaSzstek,MarcinPiotrowski,
- what is agood and a bad invoice?
- what happens to an invoice once it arrives?
- what is the best way to pay an invoice?
- how should the email accompanying an invoice look like?
- which elements should actually stay on an invoice?
- what should happen if you forgot to pay?
- technology probes
- - simple, exible, adaptable interactive installations - combining collecting data about the use of the technology in a real-world setting, testing the technology and inspiring users (and designers) to think of new kinds of technology - installed in user environment with the goal to observe them over a period of time - helping to identify design partners determining which kinds of technologies would be interesting to pursue - open-ended, inspiring new activities and encouraging experimentation
- managing interruptions | project supervised by Aga Szstek
- dealing with stress at work
- design probes
- - collaboration with a singular person based on a very deep engagement in the topic that is personally signicant - designing objects, which reect individual experiences and their meaning for the user - objects that support reection and auto-reection - such objects often become a part of an intimate relationship between the designer and the user
- Top Trumps probe cards to mark objects which are perceived to have magical power
- helping to keep a straight posture | Magda Rydygier project supervised by Aga Szstek
- preparing the study - brainstorm the context to the given problem (e.g. through contextmapping exercises) - brainstorm on user expectations and the ideal experience - prepare a list of questions regarding the current experience - decide on the format of the cultural probe (digital elements, physical element and intangible elements) - select the criteria for choosing participants and prepare awards - develop the probes and the instruction - organize a session with participants to explain to them the elements of the probe and the study procedure - provide informed consent forms - personally give the probes to the participants
- running the study - the study typically lasts no longer that one week (but this is very individual to the research problem) - usually the study happens without an active involvement of the researcher - if the study involves an online diary check the entries every day and comment on them - give participants a phone number and an email address where they can contact you in the case of any problems - at the end of the study invite all participates either to a joint session (a Focus Group or a creative workshop) or for individual interviews - record the sessions for later reference
- summary - the research method should reect the information you are trying to obtain - dont be afraid to combine the methods and experiment with them - think of creating visual materials for your studies they engage the users bringing you more insight - user research is a moment when you approach users with open mind try not to bias them and put your ideas into their mouth - the quality of the data you collect will likely be directly reected in the ideas you will create later on
- references Gaver, William W., et al. "Cultural probes and the value of uncertainty." interactions 11.5 (2004): 53-56. Gaver, Bill, Tony Dunne, and Elena Pacenti. "Design: cultural probes." interactions 6.1 (1999): 21-29. Mattelmki, Tuuli. Design probes. Aalto University, 2006. Hutchinson, Hilary, et al. "Technology probes: inspiring design for and with families." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2003. Buijs, Jan. The Delft innovation method: A design thinker's guide to innovation. Eleven International Publishing, 2012.