Tag: information architecture

Card Sort Reflections & Analysis

In July, I had done a card sort study for the section of the website I was helping to redesign.  Particularly since the new portal I’ve been working on doesn’t have as clear cut categories, we decided to do another card sort.

Reflections
Just a Few Number of Sessions worked fine.  The first time we did the study, we did 5 group sessions and found that we began finding the same results, especially after refining it the first time.  We only did 4 group sessions this time and we still found after the 3rd session, we found nothing new (though that may have had something to do with the make-up of the 4th group).

Timing was an issue. Although it was somewhat an issue the first time too (because it was summer), but this time was almost worse because I had less time between advertising and carrying out the study.  And, although there were a lot more people on campus, the study was carried out around midterms.  Thus, it was even more difficult to schedule people into the same times.

Advertising online worked 100x better whether it was e-mailing certain mailing lists, posting on the psychology department’s list of surveys, or e-mailing previously interested people who’s schedule just didn’t work with ours for the first study versus posting paper posters around campus.

Getting people to think in the right mind frame was again an issue. I won’t go into this too much though it was interesting that I found students to have less problems with this than those who worked on campus.  I will not even begin to theorize why particularly since that was a trend over only 9 groups of participants.

Participants can be a great source. As we were doing another closed card sort, we had pre-set categories, but one of the participants in the first group came up with a much better categorization by adding a couple of categories, while removing one, creating less ambiguous categorization.

Analysis
As I didn’t write about this last time, I thought I’d write a little bit about analysis this time (I used the same method).  After gathering the results (simply by writing down the numbers of the sticky notes), I entered them into xSort, a free MAC card sort statistical program.  The program also allows sessions for participants to enter data, but is designed for individuals rather than groups, so I opted to put in the results myself and using it primarily for analysis.

Statistical Analysis
The program provided the standard distance table and cluster tree results.  The cluster tree options included single, average, and complete linkages.  From what I have read of the literature, it seems as if using average linkage trees is the most common and I did find that single linkage gave many more branches (and generally more groups too), whereas complete linkages gave few groups but also many more outliers when using a cut off in the standard range of 04.-0.6.  Average linkage gives a good balance between the two, but of course, I did not simply take the cluster tree and turn that into a new IA.

Subjective Analysis
During the study, I had also taken a lot of notes on labels that participants found problematic and their suggestions.  I also took notes on item categorization that participants found difficult to put into a single category, which was generally reflected in the cluster tree as well by tending to be the outliers or items that were not categorized.

Using the Results
Using the average link cluster tree, I used that as a basis for an IA. Many of the problematic labels identified in participants’ comments were renamed to better reflect the content that a link would point to, which also helped putting them into the right category.  One link we ended up never putting into a category and decided to work it into the design outside of the categories that we had created.  This version of the IA was then put forward as a draft which will hopefully see little change before the “final” version is made for the portal.

Inventory & Not Reinventing the Wheel to Create an IA

I had previously written about creating an IA basically through inventorying an existing site and using some basic assumptions to choose what to include.

I was recently tasked with creating another new section or portal to the website, but this time, I was not working off of an existing section.  Instead, I am creating a new section based on our needs and what other similar organizations have done.  So, this time I did it differently. In a sort of two step process:

  • inventory
  • looking at other websites

The websites I looked at were actually chosen by my boss because he knows which ones generally had the resources to do a lot of testing with their users and a good IT department with experienced staff members (or maybe it was just that he found these ones to be really good, probably both).  Looking at other websites helped create some initial categories as well as identify items that we might have missed in our inventory since there was no easy way to search for the content we needed.

Based on logical groupings and categories that other sites used, I created an initial IA to be used as part of the card sort study.

Card Sort Methodology

So recently, I’ve been working on a mini-usability design study by asking users to do a card sort. In the process, I found some interesting tidbits.

What’s a Card Sort?
For those who don’t know what a card sort is, you basically put ideas (i.e. possible links to pages) on index cards or sticky notes and ask people (usually in a group) to sort them into categories, either existing ones you provide or ones that they name after.

Number of People to Test
Interestingly, I found that some articles suggested 25-30 people, but according to Nielson‘s correlation study, 15 is enough and after 20, it’s not worth the resources.

Card Sort Methodology
Open-sort vs. Closed-sort: We decided to use a close sort (categories are pre-determined) since we had already created a proposed information architecture (i.e. navigation structure).
Group vs. Individual: I had originally planned to do individual sessions since that would be more flexible, but J. (a coworker) has read studies about how these sorts of exercises work better in a group. I have read in various articles that group card sorts is the preferred method, so that made sense.
Silent vs. not: J. also suggested a silent card sort, which really did affect the group dynamic. I could see that even when silent there were people who were more assertive than others and that during the discussion that followed, those people were definitely more opinionated as well. So, I’m glad we did it as a silent sort.

Reflections
Scheduling was definitely much more time consuming than I had thought it would be. And trying to find faculty was the most difficult. Perhaps due to the incentive that we provided ($10 for 30 mins), we had plenty of student volunteers, especially grads (probably because they were around whereas undergrads were less likely to be as it’s between the two summer terms). For faculty, our hardest-to-get group, personal e-mails were definitely necessary! (and from someone they know).

Getting people to think in the right mind frame was also an interesting task.  A number of people who participated kept thinking about the design. Although it brings up interesting points which are helpful while we design a new site, some of it was irrelevant. Some kept thinking that it would be the home page, but no… it is not. They got the idea that what they saw was definitely going to be on the website, but that’s not true either. It got a bit frustrating at times, because I would basically say, “yes, good point, but let’s focus on the task at hand” (which was the card sort itself and naming the categories).  Most of the time it worked, but with one or two people… somehow that didn’t. They were so focused on “this is what and how I would like to see the website to be”, so I had to repeat more than once that it’s not the home page, just a single page somewhere. I got around it by turning my open questions into closed questions, but man… argumentative people can definitely change the group dynamics and totally veer the discussion in a totally different direction. Okay… apologies… </rant> But I think it brings up the important point that having a good mediator/facilitator is very important. I honestly think that my coworker would have done a better job than I did, but ah well, you do what you can.

Backup plans are a must-have! What if something goes wrong? Terrible on my part, I know, I did not really think about it before the actual sessions took place. What do you if someone doesn’t show up? What if more people suddenly show up? Does it matter to your study? I decided that for our purposes, if one person give or take in a group wasn’t a big deal, but definitely something to think about next time. Making sure you have all needed materials and back-up materials if things break down is also another much needed consideration.

Another Online Resource
Finally, there were a lot of good online resources. In particular, Spencer & Warfel’s Guide is quite comprehensive.

One Method to Create an IA

Information Architecture, or IA, is just IT jargon for an organization or navigation system, typically a hierarchy. Actually, it depends who you ask. For a general idea on why this definition might be contentious, please google it, or refer to the Wikipedia article on IA. Anyhow, for the purposes of my post, please think of the IA as I have defined it, which is what I was told in my department.

Gather Data
I first gathered a lot of data (as the previous post probably indicates). What I basically did was do an inventory of existing content. I also looked at each page to see what topic it generally covered. In doing the inventory, I think came up with three major data tables.

  1. How many pages covered the same topics.
  2. Which pages showed up in the inventory more than once.
  3. Which pages were visited the most (of what I included in the inventory).

Assumptions to Think About
Oh yes, of course, using this kind of data has certain assumptions of course, which may or may not be true.

  • The more pages on a topic that staff have created, the more important staff think this topic is.
  • The more a page is linked to, the more staff find it a useful page.
  • The more a page is visited, the more useful that page is to our users.

Although I don’t think these assumptions should be accepted without any scrutiny, I think they’re also okay to make, to a certain extent. These assumptions aren’t always true, but they can be used as indicators. Also, I would be hard pressed to use only one of these indicators, but in combination can be used as a good base.

Creating the Base IA

  1. Summarize: Using the 3 tables, I did a sort of summary table by way of ranking and counting to come up with the most common topics.
  2. Group: I then grouped them in such a way that it made sense to me and named each group.
  3. Fill in the Holes: I filled in any “holes”, such as in the category Finding (Library Resources), Books and Journals were obviously in there, but Maps were not.
  4. Add what’s Missing: I think consulting with one or more expert (in this case, librarian) is a really important step in recognizing that there may be really important pages that people just can’t find or don’t know exist (or maybe doesn’t exist yet).

Work in Progress
No wonder programs go through so many versions, I don’t know how many I’ve gone through just consulting with one other person!  No doubt it’ll go through many more as users and other staff are consulted. As long as it doesn’t degrade into this: How a Web Design Goes to Hell