[Abstract]
In a world where technology is readily available to us in all shapes and sizes, how does device size affect our ability to accomplish certain tasks? Processing speed, input method and screen size have all been studied as factors that affect the user. However, taken altogether, how will these factors affect a search and selection task, and why? This paper will specifically examine the differences between a mobile device and a computer with a large monitor. The differences in task performance on the different devices could have important implications for which tasks we choose to perform on which devices.
[Hypotheses & How they will be measured]1. Mobile device users will use more category heuristics than large-screen users.
This is to be measured based on how many times the user refines his search on a given task. For example, if a user searches for "Summer Dresses > Floral" under Dresses, that is a value of '2' for the number of heuristics used to refine the basic search.
2. Mobile device users will be more satisfied with their selection because they had fewer choices.
The reasoning behind this hypothesis is that the mobile device users will be able to view fewer dresses, both because the processing/network speed on the mobile device is slower and because the screen is smaller. The large screen user will be able to view many more options. Thus, based on the paradox of choice, the mobile device user will be more satisfied choosing from X options whereas the large screen user will experience decision paralysis choosing from Y options (where Y >> X). There is one confounding variable in examining the paradox of choice in practice with this particular experimental set up: time. Users on the mobile device may be frustrated by the fact that though there are Y potential choices, he can only access X of them. This is to be measured based on pre and post survey questions.
3. Large-screen users will perform better, as rated by independent raters.
This is to be measured by having mechanical turkers rate the subjects' selections based on how well they think the selection fits the criteria presented in the prompt. I am also considering having 'fashion experts' rate all of the selections. This would give me multiple modes of measurement with which to confirm this hypothesis.
4. Mobile device users will feel like they need more time more than large-screen users.
I have selected 5 minutes per task, for a total of 10 minutes, as the allotted time for this experiment. This seems to be appropriate based on my pilot experiments in which some participants ran out of time and some did not. I am keeping track of how how long it takes each participant in each task to complete. I plan on comparing on a binary scale (needed more time---did not need more time) but I will have the data to do more involved time analysis if this becomes desirable.
[Methods]
See Google doc for pre & post task survey as well as task instructions.
1. I have altered the participant instructions to just leave the tab open to his/her selection when done and I will record this later. This allows me to have the participant complete the survey on the computer but just perform the task on a mobile/large device.
2. I also updated the instructions to include a more specific description of Jamie and Matt and what they are looking for:
Excerpt
[Task 1]
Read below for what kind of dress Jamie would like:
"I'm going to my friend's wedding in July and I want to find a fun, flirty dress. I am 5'8" with a pear-shaped build. I have brown hair and olive complexion. I want something that will look hot, but not distract from my friend's big day, nothing too flashy! Help me!"
[Task 2]
Read below for what kind of shirt Matt would like:
"I'm a little bit anxious about this pool party. I want to wear a shirt that will look casual, but make me seem in style. I don't usually get too hot in the sun, so I think I'd rather cover up than bare my arms. I'm 5'10" with dirty blond hair and freckles. I have the lean build of a distance runner. Make me look good!"
3. I am offering prizes as follows: "There will be a prize for the best, most appropriate dress selection and shirt selection. The winners will get their choice of a $5 Starbucks, Jamba Juice, itunes or Philz gift card. If the same person wins both dress and shirt, he/she will get a $15 card. "
My reasoning for giving more if someone wins both shirt and dress is to incentivize someone to try hard on both tasks and overcome task fatigue.
[Materials]
I am using a iphone 4G as my mobile device and 27" monitor as my large-screen.
I am using safari web browser for both conditions.
[Further Questions] **If anyone is commenting on my blog, advice on this area would be particularly desired!
1) I'm still not sure how I can watch/observe my participants on their mobile devices. I would have to be a huge creeper!
2) I am not sure exactly how I want mechanical turkers to rate the selections. Rank order? Rate each individual on a scale or multiple scales? A 'which is better' A/B comparison?
What I need to do: