To determine the extent to which the text of menu items articulates the structure of the site, and clarifies what the target pages are about.
The tactics for creating effective menus appear here as a set of guidelines based on research and actual experience, documented in books, journal articles, and online styleguides. (References appear at the end of each strategy). Each guideline, then, provides a method for a writer to follow, or a heuristic.
In this evaluation we test the text against these guidelines. You are, then, performing a heuristic evaluation.
Here’s how to perform a Heuristic Online Text (HOT) evaluation.
1. Save this file with a name that includes
q The site you are analyzing
q The aspect you are evaluating (menus, in this case)
q Initials
q A period
q A suffix indicating the file type (doc for Word files, htm for HTML files)
Examples: ibmmenusjp.doc,
yahoomenusds.htm
2. Go to a site’s home page, and reproduce the home
page menus in outline form.
You may find a set of menus at the top of the page, another set on the left, a third in the middle, a collection of extra references on the right, and a reprise of most of the major items at the bottom of the page.
Please try to make sense out of all the menus. The top level of your outline is Home Page. But what comes below that?
· Usually menus across the top are more important than any others.
· Check the menu at the bottom of the page for indications of the topics that the site designers consider most important.
· If you have trouble identifying the relationships between the various menus, please add descriptions (in parentheses).
· Isolate the menu at the bottom of the page, labeling it “Bottom Menu.”
3. Follow one of the major menus down, down, down,
recording several more levels in your outline.
Use indentation to indicate a lower level.
Reproduce the actual text of menu items, in order.
4. Return to the home page and copy the URL for
that page, then paste that into this file, in the line right after the last
item in your outline.
The URL is the address of the home page.
5. Type today’s date on the next line, to show when
you collected the menu items in your outline.
6. Apply the HOT Evaluation to the outline you have
collected, filling out the evaluation form.
If a strategy or tactic seems irrelevant, omit it from your evaluation. Note that this will change the total possible points.
URL to the page:
Date investigated:
Test
Clicking the linktext of each menu item takes me to a
page in which that text reappears, verbatim, as the title or major heading on
the page, confirming that I have gotten where I expected to. The page title or
major heading is the same object used in the menu, as a menu item.
YES=1, NO=0.
Impression
Overall, the linktext of menu items seemed similar to
that text of the title or major heading on target pages.
YES=1, NO=0.
Example
Which menu item seemed particularly out of sync with
the title or major heading on the target page, if any?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
Tests
Comparing all the items in a single menu, I can tell
the difference. There are no items that seem as if they might be describing the
same topic.
YES=1, NO=0.
When I think of a topic I would like to find through
the menu system, I can figure out which menu item might lead me toward that information.
(Try 5 topics).
YES=1, NO=0.
Impression
Overall, the menu items are easy to tell apart.
YES=1, NO=0.
Example
Which menu items seemed as if they might be pointing
to the same content?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
Test
Reading almost any menu item, I can correctly guess
what information will appear on the target page.
YES=1, NO=0.
Impression
Overall, the menu items are accurate and complete
enough to let me gauge what content will appear on the target page.
YES=1, NO=0.
Example
Which menu items seemed inaccurate, or inexpressive?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
Test
In looking at ten menu items that contain more than a
few words, I see that the keyword always appears early in the item, not late.
YES=1, NO=0.
Impression
Overall, I can distinguish between menu items because
the important words appear early in each item.
YES=1, NO=0.
Example
Which menu items seemed to leave keywords to the end?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
Test
On the home page, and major departmental pages, I
always find links to popular pages that exist deep within the site.
YES=1, NO=0.
Impression
The site is helpful, offering menus leading to
important pages that lie deeper within the site (not at the next level down).
YES=1, NO=0.
Example
Which pages were pointed to? If none, which pages do you think the site should have offered
shortcuts to?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Conklin (1987), Cooper (1995), Farkas and Farkas (2000), Keeker (1997), Mandel (1997), Shneiderman (1992), Shneiderman and Kearsley (1989) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
Test
Each menu of less than seven items contains menu items
written so that I can tell why these items appear in a group, and why they
appear in this sequence.
YES=1, NO=0.
No short menu is arranged in alphabetical order,
unless it is a list of items that usually appear in that arrangement (states,
authors’ names, book titles).
YES=1, NO=0.
Impression
Overall, I can grok why these items go together in
this order.
YES=1, NO=0.
Example
Which menu items seemed out of place in a short menu?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
Test
Each menu of more than seven items is broken into
groups.
YES=1, NO=0.
When menu items appear in a group, I can see that the
items all refer to the same kind of object, category, service, product, or
activity. I see why all, or almost all of them belong together.
YES=1, NO=0.
Within each group of menu items, I can see, from the
text, why they are arranged in this sequence. I can at least grasp why one item
is first, others are in the middle, and another comes in last.
YES=1, NO=0.
When I look at the sequence of groups, I can see some
familiar pattern emerging. I am not
puzzled by the order of the groups.
YES=1, NO=0.
No long menu is arranged in alphabetical order, unless
it is a list of items that usually appear in that arrangement (states, authors’
names, book titles)..
YES=1, NO=0.
Impression
Overall, I can spot a meaningful arrangement of the
menu items in groups.
YES=1, NO=0.
Example
Which menu items seemed to out of place?
Which groups seemed out of place?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Abeleto (1999), Ameritech (1998), Apple (1987), Gregory (1987), Hix & Hartson (1993), Krug (2000), Larson & Czerwinski (1998), Lynch (2000), Mandel (1994, 1997), Sullivan (1998) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
Test
There are distinct menus for several different
audiences. But the content is not entirely separate. Following the trail of one
audience, I might find the same page I already viewed when I posed as another
audience.
YES=1, NO=0.
There are several menus set up using different
categories (toys by age, toys by type, toys by character, toys by vendor). I
could get to the same page from several different menus. (Not necessarily from
every menu).
YES=1, NO=0.
Impression
Overall, I feel that the multiple menus reflect the
different interests, perspectives, or mental models of typical audiences.
YES=1, NO=0.
Example
Which page could you find from several different
menus?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Bushell (1995), NCSA (1996), Veen (2001) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
Test
The second-level items appear at the same time as the
main items, and stay put as long as I am on the page.
YES=1, NO=0.
If the second-level items appear after I select
a major item, they stay put, and do not disappear when I move the mouse, or
release my grip on the mouse button. They remain stable.
YES=1, NO=0.
Second-level menu items never appear in
cascading menus (temporary dropdown menus that go away if you slide your mouse
the wrong way). Second-level menus never appear when I hover over the
right areas, disappearing when I jiggle the mouse.
YES=1, NO=0.
The site allows me to explore more than two levels of
menu items, all on the same page, so I can get a sense of the full content of
the section before proceeding.
YES=1, NO=0.
Impression
Overall, I can see several levels of content in the
menu items displayed on most pages.
YES=1, NO=0.
Example
Which page offers the most levels of menu items? (Insert a screenshot).
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Cooper (1995), Mandel (1997), and Norman (1991) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
Test
I know that I have arrived at the right page, because
the text at the top of the page is the same text I just clicked.
YES=1, NO=0.
Some or all of the words in the title or major heading
appeared in the menu item.
YES=1, NO=0.
The introductory sentence echoes those words, or the
same idea.
YES=1, NO=0.
The caption of the first big picture echoes those
words, or the same idea.
YES=1, NO=0.
Impression
Overall, I can confirm that I have arrived at the page
I expected to.
YES=1, NO=0.
Example
Which page surprised you the most, because it was not
what you expected when you clicked the menu item?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Apple (1999), Krug (2000), Lynch and Horton (1999), Microsoft (2000) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
Test
The page displays breadcrumbs, or some other text or
graphic device showing where I am in the larger structure, for instance,
displaying the major choices I would have made if I came down from the top,
following a common path.
YES=1, NO=0.
Impression
Overall, I can see where I am within the larger
structure.
YES=1, NO=0.
Example
Which pages completely lacked any indication of
location?
Comments
Please comment on the ways in which the text of menu
items seems to follow, or ignore, this guideline. Please record what you have
observed, while doing the testing.
See: Apple (1999), Black & Elder (1997), Bransford and Johnson (1972), Farkas and Farkas (2000), Keeker (1997), Nielsen (1996), Omanson et al (1998), Siegel (1996) in the bibliography at http://www.webwritingthatworks.com/HTres2cbiblio.pdf.pdf .
.
Assigning a grade to a set of texts is always a bit arbitrary. But counting up the points for these sample texts, we reach this diagnosis:
Total Points:
Total Possible:
Percentage:
90-100%: Excellent menus.
75-89%: Pretty clear menus, but occasionally confusing.
60-74%: Could use some rewrites, and reorganization.
45-59%: A mess.
25-44%: Sure to confuse visitors.
0-24: Guaranteed to make visitors reach for the Back button.
In a few paragraphs, summarize your most important observations—both
positive and negative.
List the top five problems with the menus, and in a sentence or two,
summarize what you would recommend as solutions.