I’ve been working on our new website and as usual, I decided to start with a card sort. Continue reading “Card Sort & Goal-Oriented IA”
So this time I did a proper card sort, because the services section is a lot bigger than the others. How to group services and the pages therein is also a lot less clear cut. Continue reading “Services Card Sort & Revised IA”
As part of the ongoing effort to improve the website, I have been redesigning the website one section at a time. Earlier in the term, I did the Research Help section. In the last couple of weeks (that’s right, only about 2-3 weeks!), I worked on getting the ‘Articles’ section cleaned up with a new landing page.
Card Sort to Clean Up
I knew that I wouldn’t have time to do a proper card sort study, but I decided to do a card sort with the web committee. We have enough people that I divided the committee up into two groups and gave them sticky notes with the names of the existing pages.
The sticky notes consistent all the local navigation links from the left side of the existing page and the second level links you see in the expanded part of the ‘More’ menu here (right). Then, I gave the usual instructions for a silent open card sort (no talking, group as you see fit).
While the purpose of a card sort is not to help clean up a section, it really got staff to see what fit in the Articles section and what didn’t. A lot of the pages were also deemed no longer relevant.
As a group, we made decisions on what to remove, and what to move and where. Quite a lot of the content was moved to either our subject research guides or our FAQ system.
There were one or two pages that we couldn’t decide what to do with, so for the time being they’ve been left where they are for now (e.g. RefWorks page is still in the ‘Articles’ menu though not linked on the landing page).
With the pages that were left, I created a few mockups.
The web committee met again to discuss the mockups. I already had my heart set on either #2 or #3, because the whole idea is that it’s simple and clear. Having only one search bar with just a few links give users focus on what they’re looking for. Different people had different preferences on mockups, but with some discussion, the group settled on mockup #3 with a few revisions.
As with the research help page, I posted the revised Articles mockup in the staff room to give everyone a chance to provide feedback. This time, I didn’t get any feedback that resulted in any changes, so the final page is the same as the revised version.
Even though I’m on vacation, I wanted to get the new page up before the next term, so the new ‘Articles’ landing page went up this week.
In July, I had done a card sort study for the section of the website I was helping to redesign. Particularly since the new portal I’ve been working on doesn’t have as clear cut categories, we decided to do another card sort.
Just a Few Number of Sessions worked fine. The first time we did the study, we did 5 group sessions and found that we began finding the same results, especially after refining it the first time. We only did 4 group sessions this time and we still found after the 3rd session, we found nothing new (though that may have had something to do with the make-up of the 4th group).
Timing was an issue. Although it was somewhat an issue the first time too (because it was summer), but this time was almost worse because I had less time between advertising and carrying out the study. And, although there were a lot more people on campus, the study was carried out around midterms. Thus, it was even more difficult to schedule people into the same times.
Advertising online worked 100x better whether it was e-mailing certain mailing lists, posting on the psychology department’s list of surveys, or e-mailing previously interested people who’s schedule just didn’t work with ours for the first study versus posting paper posters around campus.
Getting people to think in the right mind frame was again an issue. I won’t go into this too much though it was interesting that I found students to have less problems with this than those who worked on campus. I will not even begin to theorize why particularly since that was a trend over only 9 groups of participants.
Participants can be a great source. As we were doing another closed card sort, we had pre-set categories, but one of the participants in the first group came up with a much better categorization by adding a couple of categories, while removing one, creating less ambiguous categorization.
As I didn’t write about this last time, I thought I’d write a little bit about analysis this time (I used the same method). After gathering the results (simply by writing down the numbers of the sticky notes), I entered them into xSort, a free MAC card sort statistical program. The program also allows sessions for participants to enter data, but is designed for individuals rather than groups, so I opted to put in the results myself and using it primarily for analysis.
The program provided the standard distance table and cluster tree results. The cluster tree options included single, average, and complete linkages. From what I have read of the literature, it seems as if using average linkage trees is the most common and I did find that single linkage gave many more branches (and generally more groups too), whereas complete linkages gave few groups but also many more outliers when using a cut off in the standard range of 04.-0.6. Average linkage gives a good balance between the two, but of course, I did not simply take the cluster tree and turn that into a new IA.
During the study, I had also taken a lot of notes on labels that participants found problematic and their suggestions. I also took notes on item categorization that participants found difficult to put into a single category, which was generally reflected in the cluster tree as well by tending to be the outliers or items that were not categorized.
Using the Results
Using the average link cluster tree, I used that as a basis for an IA. Many of the problematic labels identified in participants’ comments were renamed to better reflect the content that a link would point to, which also helped putting them into the right category. One link we ended up never putting into a category and decided to work it into the design outside of the categories that we had created. This version of the IA was then put forward as a draft which will hopefully see little change before the “final” version is made for the portal.
So recently, I’ve been working on a mini-usability design study by asking users to do a card sort. In the process, I found some interesting tidbits.
What’s a Card Sort?
For those who don’t know what a card sort is, you basically put ideas (i.e. possible links to pages) on index cards or sticky notes and ask people (usually in a group) to sort them into categories, either existing ones you provide or ones that they name after.
Number of People to Test
Interestingly, I found that some articles suggested 25-30 people, but according to Nielson‘s correlation study, 15 is enough and after 20, it’s not worth the resources.
Card Sort Methodology
Open-sort vs. Closed-sort: We decided to use a close sort (categories are pre-determined) since we had already created a proposed information architecture (i.e. navigation structure).
Group vs. Individual: I had originally planned to do individual sessions since that would be more flexible, but J. (a coworker) has read studies about how these sorts of exercises work better in a group. I have read in various articles that group card sorts is the preferred method, so that made sense.
Silent vs. not: J. also suggested a silent card sort, which really did affect the group dynamic. I could see that even when silent there were people who were more assertive than others and that during the discussion that followed, those people were definitely more opinionated as well. So, I’m glad we did it as a silent sort.
Scheduling was definitely much more time consuming than I had thought it would be. And trying to find faculty was the most difficult. Perhaps due to the incentive that we provided ($10 for 30 mins), we had plenty of student volunteers, especially grads (probably because they were around whereas undergrads were less likely to be as it’s between the two summer terms). For faculty, our hardest-to-get group, personal e-mails were definitely necessary! (and from someone they know).
Getting people to think in the right mind frame was also an interesting task. A number of people who participated kept thinking about the design. Although it brings up interesting points which are helpful while we design a new site, some of it was irrelevant. Some kept thinking that it would be the home page, but no… it is not. They got the idea that what they saw was definitely going to be on the website, but that’s not true either. It got a bit frustrating at times, because I would basically say, “yes, good point, but let’s focus on the task at hand” (which was the card sort itself and naming the categories). Most of the time it worked, but with one or two people… somehow that didn’t. They were so focused on “this is what and how I would like to see the website to be”, so I had to repeat more than once that it’s not the home page, just a single page somewhere. I got around it by turning my open questions into closed questions, but man… argumentative people can definitely change the group dynamics and totally veer the discussion in a totally different direction. Okay… apologies… </rant> But I think it brings up the important point that having a good mediator/facilitator is very important. I honestly think that my coworker would have done a better job than I did, but ah well, you do what you can.
Backup plans are a must-have! What if something goes wrong? Terrible on my part, I know, I did not really think about it before the actual sessions took place. What do you if someone doesn’t show up? What if more people suddenly show up? Does it matter to your study? I decided that for our purposes, if one person give or take in a group wasn’t a big deal, but definitely something to think about next time. Making sure you have all needed materials and back-up materials if things break down is also another much needed consideration.
Another Online Resource
Finally, there were a lot of good online resources. In particular, Spencer & Warfel’s Guide is quite comprehensive.