Capstone: Evaluating a Real Interface End to End

• Plan an evaluation that states its users, tasks, context, and what it does not cover.
• Combine task analysis, heuristic evaluation, a usability test, and a WCAG check, and say what kind of evidence each one gives.
• Separate what a tool measured, what an evaluator judged, and what only a test with people can show.
• Turn mixed findings into one prioritized list with severity, evidence, and a fix.
• Report the limits of an evaluation as plainly as its findings.

A list of problems with no evidence behind it gets argued away, and an accessibility score with no task behind it gets mistaken for usability. This capstone applies the methods of the discipline to one real interface: the FP-Growth lesson page of this site, https://university.kurz.fyi/computer-science/ai-theory/data-mining/fp-growth, as it stood on 2026-11-04. Readers can open the page and repeat the checks. The site may have changed since, because an evaluation is a dated statement about one version.

The limits come first, because they shape everything below. One evaluator made one pass. No participants took part. No screen reader was run. The scope is English, the light theme, and a signed-out reader.

Four methods, four kinds of evidence

The methods overlap little, which is the reason to combine them. In this evaluation the tool found the blocked zoom and missed the horizontal overflow, and neither the tool nor the heuristics can say whether a reader understands the page.

Label the evidence

Every line of a report should say where it came from. This one uses three labels: measured (a script or tool returned it), judged (an evaluator's reading of the interface), and fictitious (invented to show arithmetic). A measured line can still be unimportant and a judged line can still be right, but a reader has to be able to weigh them differently.

From severity to priority

Severity on Nielsen's 0 to 4 scale combines frequency, impact, and persistence, and ratings are stronger when several evaluators give them. Priority adds cost: of two problems of equal severity, the cheap fix goes first. A requirement can override both, for example when a contract or a law asks for WCAG AA.

Step 1: scope

• Users: a student reading alone, as in what-hci-studies-users-tasks-and-context, on a laptop or a phone.
• Goal: reach and read the worked examples of a lesson whose topic is known.
• Context: 1280 by 800 on desktop, 390 and 320 wide on a phone, signed out.
• Not covered: signed-in features, the dark theme, tablets, and pt-BR beyond one automated run.

Step 2: task analysis

Hierarchical task analysis, with plans:

text
0. Reach the worked examples of FP-Growth     Plan 0: do 1, then 2
1. Open the lesson                            Plan 1: if the name is known, 1.1; otherwise 1.2
   1.1 Search: open with Cmd+K, type "fp-growth", press Enter
   1.2 Browse: from /computer-science, open Data Mining, open FP-Growth
2. Reach the section: scroll to "Worked Examples"

Measured. Path 1.1 took 11 key actions: the chord, 9 letters, and Enter. The first result was preselected and was the right lesson. Path 1.2 took 2 clicks. Step 2 is where the cost sits:

text
viewport     page height   "Worked Examples" at   screens to scroll
1280x800          8012 px              2410 px                 3.0
390x844          13687 px              4523 px                 5.4

The page has no contents list, and its h2 elements have no id, so a reader can neither jump to a section nor link to one.

Step 3: heuristic evaluation

Judged, by one evaluator in one pass, so the severities are one opinion, and nielsens-ten-heuristics-and-heuristic-evaluation explains why several evaluators find more. The evidence for each line is measured or was read from the page.

Good findings belong in the report too. Search handles the keyboard well, the shortcut hint shows ⌘K on Apple keyboards and Ctrl elsewhere, and the sidebar shows the reader's place among the 16 lessons.

Step 4: the usability test that was not run

No test ran for this lesson, so nothing here reports behavior. This is the plan a team would run. It is formative: five readers from the target group, concurrent think-aloud, and three tasks.

1. Find the lesson that explains lift.
2. From there, find the example that computes it and say what the number means.
3. Share the location of that example with a classmate.

Task 3 tests H4 directly. The measures are success, time, errors, and SUS. Five readers find most problems only under the assumptions of usability-testing-and-the-system-usability-scale, so a rare problem can still be missed.

To show the arithmetic of SUS, here are five sets of responses. They are fictitious: nobody answered them, and no finding below uses them.

text
       items 1 to 10         SUS
R1     4 2 4 2 4 1 4 2 4 2   77.5
R2     3 3 3 2 3 2 4 3 3 3   57.5
R3     5 1 5 2 4 2 5 1 5 1   92.5
R4     3 2 4 3 3 3 3 2 3 4   55.0
R5     4 2 4 1 4 2 4 2 4 2   77.5
mean 72.0, s 15.7, 95% interval 52.6 to 91.4

Even with real answers, an interval this wide includes the often cited mean of 68, so five people could not separate this page from an average one. SUS here would add a rating, and the observations would matter more.

Step 5: WCAG check

Measured. axe-core 4.14.0 ran in Chrome with the WCAG 2.0, 2.1, and 2.2 A and AA tags, on three pages at 1280 by 800:

text
page                                         rules passed   violated
/computer-science/ai-theory/.../fp-growth              25          4
/computer-science                                      23          5
/computer-science/ai-theory/data-mining                19          3

On the lesson page axe also left 10 contrast results as incomplete, which need a person. The pt-BR lesson page gave the same four violated rules. The failures, joined with manual checks:

The formula gives 2.43 and axe prints 2.42, a rounding difference. Replacing the color with #6b7280 gives 4.63:1 on the same background.

Some checks passed. lang is right in both languages, and all 12 Tab stops measured had an outline or a shadow in computed style, which shows that a focus style exists and not that it is visible enough. A reader tabs through 11 stops before the first content link, and there is no skip link and no main landmark. WCAG accepts other ways of meeting 2.4.1, headings among them, so this is recorded as a usability problem and not as a failure.

Focus also stays on body after search navigates to a lesson. What a screen reader announces at that moment is unknown, because none was run.

Step 6: prioritized findings

Items 1 and 2 lead because they are cheap, they affect reading itself, and they fail AA. Item 7 has the same severity as item 8 and sits above it because it affects the task from Step 2 on every long lesson, and that ordering is a judgment too. The next step is a retest: run the same scripts after the fixes and confirm that the failures are gone and nothing new appeared.

• "A clean axe run means the page is accessible." axe missed the sideways scroll, the lost focus, and the 11 stops before the content. See accessibility-and-wcag.
• "Heuristic evaluation replaces testing with people." It predicts problems. Whether readers hit H3 at all is an open question here, since nobody was observed.
• "One evaluator is enough for a real report." It is enough to show the method. A team should merge independent passes.
• "Invented numbers are better than none." The SUS table above shows arithmetic and supports nothing. A report that mixed it into the findings would be false.
• "Severity is a measurement." It is a judgment made on a scale, which is why this report keeps the evidence next to it.
• "Sort by severity and stop." Without cost, a one-line fix waits behind a redesign.

This evaluation combined four methods on one dated page, and each found what the others could not. Task analysis measured the distance to the worked examples, 3 screens on desktop and 5.4 on a phone. One heuristic pass raised five problems. The usability test was planned and not run, so its SUS table is fictitious and informs nothing. axe and manual checks found failures of five criteria at A and AA, including blocked zoom and contrast of 2.43:1. The result is one list that keeps evidence, severity, fix, and cost together, and that states its limits: one evaluator, no participants, no screen reader. The next honest step is to fix, rerun, and put the page in front of readers.