A deep dive into questions by @cjforms at UxLx

  • Published on
    21-Apr-2017

  • View
    2.976

  • Download
    0

Transcript

PowerPoint PresentationA deep diveinto questionsWorkshop at UxLx 2014 led by Caroline JarrettHow to ask better questions, andhow to assess user experience using surveys1 CHI 2010 Caroline JarrettIntroductions(Im Caroline Jarrett - @cjforms)Work with your neighbourYour name and roleA random thing about yourself2AgendaIntroductionsHow to ask better questionsBreakHow to assess user experience using surveysWrap up3A survey I saw recently4How do we know its a survey?5And it continued.6Ill hand out an invitation I received recently by emailWork in pairsDecide whether it is a survey or something else7In this exercise, participants reviewed an email from a journeal that said it was a survey, but seemed to us more like a form. CHI 2010 Caroline Jarrett7AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up8There are four steps to answer a questionStep1. Read and understand2. Find an answer3. Judge the answer4. Place the answerAdapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)The psychology of survey responseThere are four steps to answer a questionStepA good question 1. Read and understandis legible and makes sense2. Find an answerasks for answers that we know3. Judge the answerasks for answers were happy to reveal4. Place the answeroffers appropriate spaces for the answersAdapted from Tourangeau, R., Rips, L. J. and Rasinski, K. A. (2000)The psychology of survey responseAre you ?11Lets review a questionThere is a question coming up on the next slideI will ask you to think about ONE of these four stepsRead and understand Find the answerJudge the answerPlace the answerPlease think about any problems in that particular step1213AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up14Legibility is also important15Hermann grid illusion CHI 2010 Caroline Jarrett15AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up1617In your last five days at work, what percentage of your work time do you estimate that you spend using publicly-available online services (not including email, instant messaging and search) to do your work using a work computer or other device?The approximate curve of forgettingAgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up19I saw this question on an employee survey20Lets create a list 21AgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up22Test with users to make sure youoffer the right answer options232425Offer the right widget to collect the answerKnowledge of what users want to tell youHow many answers?OfferWe know all the answers that users are likely to give usThey only have one answerRadio buttonsThey may have more than oneCheck boxesWere not sureText boxes26Allen Miller, S. J. and Jarrett, C. (2001) Should I use a drop-down?http://www.formsthatwork.com/files/Articles/dropdown.pdf27Grids are often full of problems at all four steps28Grids are a major cause of survey drop-out29Source: Database of 3 million+ web surveys conducted by Lightspeed Research/KantarQuoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) Usability testing of market research surveys ESRA LausanneBut its the topic that matters most30Source: Database of 3 million+ web surveys conducted by Lightspeed Research/KantarQuoted in Coombe, R., Jarrett, C. and Johnson, A. (2010) Usability testing of market research surveys ESRA LausanneAgendaIntroductionsWhat is a survey?How to ask better questionsThe steps to answer a questionImprove step 1: read and understandImprove step 2: find the answerImprove step 3: judge the answerImprove step 4: place the answerUnderstand why people answer BreakHow to assess user experience using surveysWrap up31Response relies on effort, reward, and trust32TrustPerceivedeffortPerceivedrewardDiagram from Jarrett, C, and Gaffney, G (2008) Forms that work: Designing web forms for usability inspired by Dillman, D.A. (2000) Internet, Mail and Mixed Mode Surveys: The Tailored Design MethodAn interesting subject helps in all the areas33Shared interestsinspire trustInterestingtopicstakeless effortInteresting subject =intrinsicallyrewardingWhat about response here?3435Your answers to this survey are important for our workBut whats in it for me? And Im really ready for a coffee.AgendaIntroductionsWhat is a survey?How to ask better questionsBreakHow to assess user experience using surveysWrap up36Lets start with a standard option: SUSThe System Usability Scale (SUS) was created in 1986It has been shown to be valid and reliableYou get a score between 0 and 100You can compare your SUS score with other systems37Brooke, J. (1996). SUS: A "quick and dirty" usability scale. In Usability Evaluation in Industry. P. W. Jordan, B. Thomas, B. A. Weerdmeester and A. L. McClelland. London:, Taylor and Francis.Jeff Sauro (@measuringux) has done a lot of work with SUS38http://www.measuringusability.com/sus.php.Jeff provides tools for scoring SUSHe has adapted it to websitesSUS scores are not percentagesThere are other commercial products with wider concepts of UX39SUPR-Q includes credibility and loyaltylicenced producthttp://www.suprq.comWAMMIonline serviceincludes access to standard databases (extra for SUPR-Q)http://www.wammi.comYour task: find an explanation of the difference between the European Commission and the European UnionUse this site: http://ec.europa.euDecide whether you had a good or bad experience40I will ask you to fill in SUS (original) or SUS (Sauro)41My journey into user experience started a long time ago, with usabilityThe extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)42Introduction to usability testing(c) Effortmark Limited 2004Working mostly in government, we were interested in effectiveness and efficiency43The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)But what about user experience?What about satisfaction?44The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)Picture credit: Flickr jek in the box45Satisfaction is a complex matterCompared experience to what?Resulting thoughts(nothing)IndifferenceExpectationsBetter / worse / different NeedsMet / not met / mixtureExcellence (the ideal product)Good / poor quality (or good enough)FairnessTreated equitably / inequitablyEvents that might have beenVindication / regret 46Adapted from Oliver, R. L. (1996) and (2009)Satisfaction: A Behavioral Perspective on the ConsumerExample: bronze medal winners tend to be happier than silver medal winners47Nathan Twaddle, Olympic Bronze Medal Winner in BeijingPhoto credit: peter.cipollone, Flickr Matsumoto D, & Willingham B (2006). The thrill of victory and the agony of defeat: spontaneous expressions of medal winners of the 2004 Athens Olympic Games. The first question was about rating satisfactionWhat were they asking us to rate?Just a guess from what you recall4849The challenge of UX and surveys: which bit to measure?The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use(ISO 9241:11 1998)???Some ideas about what we could measureIn the definitionGoDaddy customer supportGoDaddy as a provider of domain namesProductThis contact with help deskOverall experience of moving a domain to GoDaddyUsersWhat proportion of customers contact supportDemographics (example: type of job)GoalsReason for contacting helpReason for looking at GoDaddyEffectivenessWhether support fixed the problemWhether GoDaddy offers the right productsEfficiencyWhether it took a reasonable timeWhether the product is priced correctlySatisfactionHelpfulness of support personLikely to purchase again / recommendContext of useHome/office; alone/helpedBusiness / personal5152In the definitionInformation to help the European Commission design a better websiteProductUsersGoalsEffectivenessEfficiencySatisfactionContext of useWrite questions for each topicThen get another team to try your questionnaire53Find out about users goalsTip54Ask about recent vivid experienceTip55Image credit: Fraser SmithInterview firstTip56Test everythingTip57Caroline JarrettTwitter @cjformshttp://www.slideshare.net/cjformscarolinej@effortmark.co.uk5858 CHI 2010 Caroline JarrettMore resources on http://www.slideshare.net/cjforms5959 CHI 2010 Caroline Jarrett