DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXT WARSAW JUNE 2011

  • Published on
    11-Jan-2016

  • View
    30

  • Download
    1

DESCRIPTION

EVALUATING USER EXPERIENCE OF MOBILE, CONTEXT-AWARE APPLICATIONS IN AUGMENTED VIRTUALITY. DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXT WARSAW JUNE 2011. FINANCIAL SPONSOR: RAAK Pro scheme of:. RESEARCH PARTNERS:. - PowerPoint PPT Presentation

Transcript

Slide 1

DR. VASSILIS-JAVED KHAN WORKSHOP: PUTTING MOBILE SERVICES INTO CONTEXTWARSAW JUNE 2011EVALUATING USER EXPERIENCEOF MOBILE, CONTEXT-AWARE APPLICATIONSIN AUGMENTED VIRTUALITY# I would like to begin with introducing a scenario which frames the main idea of my presentation. Imagine a mobile phone application developer in one of his projects he wants to develop an application which enables supermarket shoppers to receive ads on their mobile phone. Design idea is to trigger ads based on the location of the shopper within the supermarket. It seems a good idea but testing this idea with actual users to find out the pros and cons of it is not a trivial task.In my talk I will present an extension to laboratory studies and an alternative to filed test studies for helping developers such as the one I mentioned to test context-aware applications. We have already conducted already 3 studies, using that setup, for evaluating the user experience of mobile, context-aware applications in augmented virtual environments1

Dr. Vassilis-Javed Khan Dr. Paul Ketelaar Dr. Marnix van Gisbergen Dr. Andrs Lucero

Dr. Koos Nuijten Arief Ernst Hhn

Nils DesleFINANCIAL SPONSOR: RAAK Pro scheme of:RESEARCH PARTNERS:

#

ME

Senior Lecturer, Academy of Digital Entertainment, NHTV Breda University of Applied Sciences,The Netherlands

Philips Research, EindhovenVodafone R&D, Maastricht

MSc., CEID, UPATRAS.GRPhD., Dept. Industrial Design, Eindhoven University of Technology, TUE.NL# INDEX

Position the approach we take compared to Lab & Field studies

Present our setup

Present 3 studies of the evaluation of a mobile, location-based app in our setup

Conclusions

Challenges / Future work

#

ULTIMATE GOAL OF CONTEXT-AWARE APPS

Provide added value to user

How to evaluate users experience while taking context into account?

Lab vs. Field test#

LAB

Highly controllableUseful for usability-related studies

Ecological valid?Context?#

FIELD TEST

Context

Difficult to conductLogistic & technical challengesDifficult to observeFew participantsQualitative

# Hansen et al. (2006) thought that finding a place to place their public displays would be trivial it actually proved to be a major problem. Trying to generalize their findings, the authors report no less than 19 items of a checklist to consider before actually deploying a prototype in the real setting. The authors categorized those items into three categories: hardware, software and user setting.7

EVALUATION IN AUGMENTED VIRTUALITY

ContextEasy to manipulateLarge numbers of participantsCan include mobile phones, tablets, other interaction devicesCan observe behaviorKhan, V.J., Nuijten, K. & Desl, N. (2011). Pervasive Evaluation Application within Virtual Environments. Proceedings of International Conference on Embedded Computing and Communication Systems, PECCS 2011, 261-264# The alternative method we are exploring is to evaluate applications in virtual environments augmented with mobile phones8MIXED REALITYReal EnvironmentAugmented RealityAugmented VirtualityVirtual EnvironmentMilgram, P. & Kishino, F. (1994) A taxonomy of mixed reality visual displays. IEICE transactions on information systems, E77-D(12)VIRTUALITY CONTINUUM

# that which is being augmented is not some direct representation of a real scene, but rather a virtual world, one that is generated primarily by computer9

OUR SETUP: CAVE

4 back-projection screens3.5 x 2.6 m.Illusion of a continuous 360o viewHead-tracking with 4 Wii-MotesUser wears a headband with infrared LEDHuman joystickV.E. created in Maya & rendered using the OGRE 3D engineTwo V.E.Supermarket,City in an islandMore to come#

#

# #

OUR SETUP

The virtual supermarket is a generic supermarket, based on common denominators found in typical Dutch supermarkets

Contains: isles, freezers, checkout counters, etc., all rendered with photo realistic graphics

Audio: typical supermarket music, typical announcements

~20% of the shelves are filled with products

Android app, Bluetooth communication with V.E.# NOVELTY

Immersive User ExperienceNearly photo-realistic vs. low resolution graphicsCAVE vs. V.E. presented on monitors

Research focused on UX, going beyond effectiveness and efficiencyi.e. perceived intrusiveness, shopping behavior, feeling of presence, etc.

Capturing actual behavior vs. opinionMarketing studies used exclusively scenario-based methods

# 3 STUDIES

Location-based ads

Study 1: 27 participants (pilot)Study 2: 70 participantsStudy 3: 70 participants

Wanted to measure: Perceived intrusiveness of adsHas been reported:Perceived intrusiveness of ads leads to ad irritation1High values of ad irritation lead to ad avoidance1

1Li Hairong, S. & Lee, J., 2002. Measuring the intrusiveness of advertisements: Scale development and validation. Journal of Advertising, 31(2):3747# 16

STUDY1Ad: Chewing gumPushControlled for:Product involvementAttitude towards advertising in generalMeasurement: Perceived intrusivenessResults: Location-congruent ads lead to less ad intrusiveness than location-incongruent ads (virtual location)GROUP1: SAW AD WHILE AT SOUP SHELFGROUP2: SAW AD AT COUNTER (NEXT TO CHEWING GUM)Hhn, A.E., Khan, V.J., Ketelaar, P., Nuijten, K., Gisbergen van, M., Lucero, A. (2011-to appear).The Effect of Location on Perceived Intrusiveness of Mobile Ads. Proceedings Chi-Sparks 2011# measured several control variables that have proven to affect the evaluation of advertising and location based advertising 17

STUDY2Differences with study 1:More participantsOpen taskExact same location for both groupsProducts on shelf changed for groupsSoup ad for both groupsHhn, A.E., Khan, V.J., Nuijten, K., Gisbergen van, M., Lucero, A., Ketelaar, P. (2011-to appear).The Effect of Location on Perceived Intrusiveness of Mobile Ads. Proceedings ICORIA 2011Same result with study 1Location-congruent ads lead to less ad intrusiveness than location-incongruent ads (virtual location)

# While standing at a counter a shopper is almost done with purchasing products and is probably on a different state while being busy in thinking, searching and choosing for products. One might hypothesize that the supermarket counter can be a location in which participants would perceive less intrusiveness compared to next to a shelf precisely because they have finished their shopping and are now in a more relaxed state. Therefore, we decided to pick the exact same location for the larger scale experiment and instead change the products that were visible on the shelf. 18STUDY3

Same setup with Study 2Ad of SoupLocation-BasedMeasurement: perceived intrusivenessDifferences:Same products on shelves for both groupsOne group saw a social recommendation along with the location-based adResults: ad with social recommendation is perceived as more intrusive

GROUP1:Along with the ad they saw a social recommendationGROUP2:Saw just the adStudy conducted by: Stephan van de Kruis# CONCLUSIONS

Location does positively affect perceived intrusiveness of mobile adsSocial recommendations would not help mobile, location-based ads

Demonstrated a V.E. where UX research of mobile, context-aware systems can be performedAdjustments of the V.E. and context are done easier, are more controllable and faster compared to an experiment in a real life settingReproducibilityIf there is an initial halo-effect it does not affect the actual perceptions# The halo-effect is generally defined as the influence of a global evaluation on evaluations of individual attributes of a person (Nisbett&Wilson, 1977) 20CHALLENGES / FUTURE WORK

Fill supermarket with a representative product set(~20K products)Head tracking with MS KinectHave a more complete phone appShop with mobile phoneInclude personalization profileAbility to interact with productsUse log and video input for analyzing behavior

Social context: AvatarsCorroborate qualitative findings of previous research endeavors & explore new research directionsValidation studies# SUMMARY

3 studies of mobile, location-based ads evaluation in augmented virtual supermarketACKNOWLEDGEMENTS

Photos & Graphics: Arief Ernst Hhn, Paula IsphordingCONTACT

khan.j@nhtv.nlvjkhan.com@v_j_khan

Research partners:Financial sponsor: RAAK Pro scheme of:

#

Recommended

View more >