Usability Test of the “Activegourmetholidays.com” web site: Potential Problems, Performance Implications and Suggested Solutions

Guglielmo Cavazza Seminar in Human Factors Professor Tyler Blake May 16, 2007 California State University, Northridge

Abstract Five people participated in a usability test of the “activegourmetholidays.com” web site. The purposes of the test were to assess how well participants understood the site's intentions and observe how they performed generic and specific tasks. Tasks included selecting any trip offered by the site, selecting a specific wine tasting trip to Piedmont, and filling out the booking form for the specific trip. Results indicated that participants overall liked the site, had minor problems in discriminating the headings listing the trips, and had some trouble filling out the booking form. Results are discussed and possible solutions are proposed.

Table of content

Introduction

…………………………………………………………….…………1

Purpose of the project ………………………………………………1 Reason to do this ……………………………………………………1 Users ………………………………………………………….……..1 Initial impression of the site ………………………………….……..2 Method ……………………………………………………………………...3 Participants…………………………………………………….…….3 Tasks ………………………………………………………….……..3 Procedures …………………………………………….…………….3 Results ……….………………………………………………….…………..4 Discussion ……………………………………………………………….….5 Results interpretation …………………………………………….…5 Things to be improved ……………………………………….……..7 Insights for future U.T. ……………………………….…………….7 Figure 1

Introductory page ……………………………………..…….8

Figure 2

Home page ……………………………………………….….9

Figure 3

Culinary and wine tasting page ………………………………10

Figure 4

Application form ……………………………………………11

References …………………………………………………………………..12

1

Intro Purpose of the project The purpose of this project is to assess the usability of activegourmetholidays.com. This web site offers off-the-beaten-path trips to Southern Europe and targets a niche market— customers looking for wine tasting tours, culinary events, and local Southern European destination sightseeing. The usability test will investigate if people can figure out what the site is and to analyze some typical tasks, measuring for response time, accuracy and subjective measurements. Tasks will include browsing around the site, selecting a general trip, selecting a specific trip, and filling out the form to book a specific trip. Reason to do this The main reasons for this usability test are to determine if the site works, how people actually use it, and if it can be improved. Secondary reasons are gaining experience in usability testing and completing a class assignment. Users The intended users for the web site include a wide variety of people. Ages may range from 20 to 65. Status includes people with working, as well, as middle and upper class backgrounds, mainly Anglophones, both families and singles. Average users are educated to well-educated, with interest in architecture, art, arts and crafts, wine tasting, Mediterranean cuisine, and those who might have been to South Europe in the past and want to holiday at local destinations away from overcrowded tourist attractions. Users’ knowledge about the internet may vary as well as the time spent on the internet. Participants who will be selected to test the web site will not be highly representative of the population target, therefore some adjustments considering participants’ lack of motivation concerning the task and knowledge about the trips will be made. Adjustments will be discussed later in the paper.

2 Initial impression of the web site The site looks well organized in sections and subsections. On the home page there’s a top tool bar grouping tours by subject, a local navigation menu listing nine levels, an introductory statement and some tour offers. The top tool bar leads to a second level that lists all the available tours grouped by subject, and eventually to a third level that lists trips individually. The local navigation bar leads to second-level pages that deal with news, contacts and useful information (some have links to third-level pages or other sites). The booking form is accessible from different site’s pages and levels. Almost every page requires scrolling down to view all the material. The site encourages browsing and lacks a search engine option. The site is well organized but the pages are characterized by a lot of noise defined by use of small fonts, long paragraphs describing tour offers and detail plans, and the use of color coding. This choice seems to contradict the browsing strategy adopted by the site implementer. If people tend to scan not to read (Krug, 2006) then having a range of from 500 (home page) to 5000 (trips listed on “culinary and wine tours” page) words per page does not facilitate searching for items. The headings do not stand out and the use of too many colors seems hard to the eye and reduces the ability to differentiate sections (Anderson, 2006). Lastly, the reservation form appears to be too long. It includes questions that are not readily answerable (to acquire the name of the trip, dates and prices you need to browse back, copy and paste the information) and do not seem to pertain to booking a trip. Focusing on these considerations, the current usability test is intended to inquire what participants think the site is about at a glance, observe how they use the site to select a trip to a specific destination in Southern Europe and how they fill out a reservation form for the intended trip. Reaction time to perform each task and a series of qualitative measures will be evaluated.

3

Method Participants Five students from California State University, Northridge participated in the usability study. Mean age = 23.4, mean hrs. spent on the internet daily = 5.8 (one outliers; 15 hrs daily). Two participants had strong computer skills and knowledge of web design applications. Two participants were tested on a laptop and three on a desk top. Three participants worked as researchers and one as a cashier. Tasks Participants were told to do three major tasks, one introductory to the site and two specific. They were asked to look at the Flash introduction and explain what they thought the site was about. The same question was posed concerning the home page to check for consistency. Then they were told to find any trip listed in the site. These three preliminary tasks were considered as a guide to the site because participants might not have been highly representative of the site’s target users in that they might lack motivation and knowledge relating to looking for an off-the-beaten-path trip to Europe. Subsequently, participants were told to find a second, specific wine tasting trip to Piedmont (Northern Italy). Finally they were asked to book the “specific trip” and fill out the reservation form. Procedures The test session started with demographic data collection. Participants were then asked to take a look at the introduction page and explain what the site was about. Response time and comments were collected. They were asked to enter the site’s home page and probed again on the same question. Response time and comments were collected. Next, participants were asked to select a trip anywhere in the site. They were invited to speak out loud if they wanted to and comments, steps and response time were collected. At this point they were told to pretend that they’d decided on a trip to Italy and asked if they could find a wine tasting trip to Piedmont (Northern Italy). Steps, comments and response times were collected. Finally,

4 participants were asked to figure out how to make a reservation for the trip to Piedmont and, once understood, fill out the booking form. Once again, comments, steps and response time were collected.

Results Intro: what the site is about? Mean RT to understand site: 28 seconds

Opinions: 4 people said food/restaurants 3 p. said Europe 2p. said travel company 1p. said accommodation 1p. said wine

Home page: what the site is about? Mean RT to understand site: 26 seconds

Opinions: 4 p were consistent with previous impression 1 person changed to “tourist places”

Find any tour within the site Mean RT to finish task: 44.2 seconds

Observations: All clicked on the top menu bar None selected the tours on the home page 2 p. clicked outdoor tours 1 p. clicked reserve your tours 1 p. clicked culinary tours 1 p. clicked comb you tours 3 p. chose Italy, 1 France, 1 Portugal 3 p. chose cooking tours 1 p. chose location 1 p. chose wine tasting

Find a wine tasting tour to Piedmont Mean RT to finish task: 1.23 minutes

Observations: 4 p. clicked on “culinary/wine tours” 1 p. looked for a search, clicked several buttons, went back to HP and selected “culinary/wine tours” 2 p. missed the heading “Piedmont” and needed help None but one p read the paragraph about Piedmont 1 p. scanned the red-font about tours and found Pied. 3 p. selected the specific tour for the picture of a man sipping wine adjacent to the Pied. paragraph None knew what Barolo is None knew what Piedmont is

Finding a booking form Mean RT to finish task: 51.6 seconds

Observations: All clicked on a selected trip All scrolled up and down scanning for key words All found a small, red-font link to the application form

5

Filling out the application form Mean RT to finish task: 3.18 minutes

Observations: 3 p. were unsure about how to answer “name of the program” 1 p. typed his own name instead 1 p. had no trouble at all All but one went back using the back arrow, copied and pasted the name of the trip 2 p. went back using the back arrow to copy the amount and 3 p. asked if they could type a fictitious amount All were annoyed about the back-and-forth task None understood that fees were in EU-€ and not US-$ 1 typed supplement fee instead of the regular fee 2 p. did not understand that dates were fixed 3 p. missed the last required-field question

Overall site impression Comments: All liked the site None were enthusiastic about the booking form Suggestions: Use scroll down menu on booking form Use radio buttons on booking form Make booking form user friendly Should avoid going back and forth (booking form) Should avoid asking useless questions (booking form.)

Discussion Results interpretation From the first two inquiries (looking at the introductory page and home page and explaining what the site is about), participants didn’t seem to understand what the site's purpose is (see figures 1 and 2). Their guesses were close but excluded the main point of the site: offering travel. Only two people understood the site’s function. All participants were able to find a non-specific trip offered by the site, though none saw the trips offered on the home page, they instead browsed through the site. It seems that the top navigation bar listing all types of travel is more visible and inviting than the small-font, word-dense paragraphs listing travel on the home page. All but two were able to find a specific wine tasting trip to Piedmont, Italy. Those who needed assistance in the task did not see the green-font heading listing “Piedmont” and

6 scrolled up and down with no result. The page (see figure 3) looks full of noise defined by use of color coding, small font and a high number of words (about 5,000). Those who were able to find a specific trip with no assistance found the task easy to carry out, though did not read the headings, but focused on the red-font tagline in the top-right or the paragraph. Participants were not comfortable in filling out the booking form. Questions are not clearly stated and confusion arises, specifically when participants are prompted to write the “name of the program”, “date of the program”, and “total cost of the program” (see figure 4). The noun “name” heading the first one might cause a Capture error – a type of slip in which a frequently done activity takes charge in place of the intended one, Norman, 1988 – and be confused with “type your own name” instead. “Date of the program” does not specify that dates are fixed but one has the option of imputing in any date, which might cause one to make the mistake of generalizing sense from a small amount of information (Norman, 1988). “Total cost of the program” is also ambiguous and people might type what they want to spend instead of the predetermined amount, confuse what’s due with supplemental fees, or mix up currencies. Typing a desired amount instead of the required one is a mistake similar to “date of the program". Typing the single-bed supplement fee instead of the corrected one is a slip that causes errors of substitution. Some of the questions listed at the bottom of the booking form are not directly related to the site and almost all participants found them irrelevant. One of these questions probes how the user has found activegourmetholyday.com and another asks what key-word the users employed when searching – both questions are marked as “required field” but none but two of the participants actually answered these. This can lead to a sequence error in which one performs all but one step of the task, and the task must be corrected. Response times collected are not indicative of good performance because there are no parameters to compare them against. Only the reaction time to fill out the booking form (TR = 3 minutes and 18 seconds) seems to indicate long completion time. Overall, participants seemed to like the

7 website, avoided reading paragraphs, scanned in looking for information, and struggled to fill out the booking form. Things to be improved The web site seems to work fairly well. Two major things could be improved: the prominence of the headings and the structure of the booking form. The pages are long and full of noise and people struggle to quickly find what they are looking for. One way to correct this is to decrease the number of words thus encouraging scanning rather than reading. Another way is to make the headings stand out more prominently so that scanning would be facilitated (Krug, 2006). The booking form should not contain unrelated questions to the task and likely to annoy people. Since the trip reservations are completed either by phone or e-mail, there is no need to bother people with questions that can be easily posed in subsequent communications. Make clearly understandable that fees are in EU-€, not in US-$ (or vice versa), and use scroll down menus at least for “name of the program” and “date of the program”. If scroll down menus are too cumbersome to use (there might be too many options to choose from) then make use of radio buttons or auto selection mode so that when clicking on a program information related to name, dates, and cost would be already placed in the fields. Some insights for future U.T. The booking form should be re-designed and tested for evaluating improvement in response time and likeability. The tagline should be re-written and tested to assess whether people will better grasp what the site is about. Minor changes considering the font and color of the headings of the trip lists’ layout should be implemented and tested to make sure that higher visual prominence will be achieved. Scrutiny of kayak problems should be considered in that those tasks causing problems that recover quickly or easily should not be changed.

8 Figure 1: Introductory page

9 Figure 2: Home page

10 Figure 3: “Culinary and wine tours” page (only a third of the page is shown)

11 Figure 4: Application form (only two third of the page are shown)

12 References Anderson, R. A. (2006). The art and technology of web design. Clifton Park, NY: Thompson. Krug, S. (2006). Don’t make me think: a common sense approach to web usability (2dn ed.). Berkeley, CA: New Riders. Norman, D. A. (1988). The design of everyday things. Broadway, NY. Doubleday Dell. Http:/www.activegourmetholydays.com