Background & Context to the Piece
I’ve been meaning to post this for a while now, especially with all the presentations on Koha and Evergreen at Access 2011. This is a copy of the final assignment from the library automation and systems class (from March 2011), which I did with a partner, so about half was written by someone else. Please note that we were not using the newest versions of either ILS and while we had access to the administrative panel, we focused on a user view (patron & staff), which means that some of the options we say ‘do not exist’ may simply be a matter of an admin setting we did not change (as it was a test server for the entire class). Many of these issues may have been fixed by now, and I actually know for a fact that one of the bugs I reported was fixed. Nevertheless, I thought this piece might be useful for those either working on or looking at these ILSs.
Original PDF of the Handed-in Version
1. Introduction
Traditionally, libraries have purchased integrated library systems (ILSs) from vendors who make proprietary software. However, open source ILSs have become much more popular in recent years, with a number of major systems and several companies offering support (Breeding, 2009). Our task was to compare two of these systems: Koha (Bywaters version 3.03) and Evergreen (version 1.6.1.2), specifically the circulation module both through the staff client and the Online Public Access Catalogue (OPAC). A number of techniques were used in the evaluation and comparison process. Our main strategy was direct examination; we performed an operation in one system and then the other, which immediately highlighted any differences. We also consulted the systems’ documentation materials, read users’ reviews, and, at times, turned to listservs and bug reports. Our findings are laid out in a combination of narrative, charts, screen captures, and bulleted lists. For each section, we have attempted to select the medium that provides the most clarity for the user.
We considered several possible approaches to the division of work, and ultimately decided that we would each be responsible for roughly half the circulation modules. This meant that each of us had the opportunity to explore both systems in depth. The remainder of the project was divided up fairly informally, but with an eye to maintaining balanced contributions.
2. Which Version of Koha?
The recent division of Koha into two development streams, ByWaterSolutions and LibLime (PTFS), is a complex and controversial piece of recent history. Tensions exist between the major support companies in the United States and Europe (Hellman, 2010), and both lines have their supporters. However, the ByWater release seems to have gained more acceptance within the Koha community (Leonard, 2010). The ByWater website’s links point back to the original Koha community, whereas LibLime’s directs the user to the koha.org site.
In addition, a large-scale survey of libraries in 2010 showed significantly more user satisfaction from libraries using the ByWater release, particularly in the area of customer support (Breeding, 2011). According to recent postings to a library listserv (Krischel, T., February 17, 2011 [Web4Lib list]), LibLime’s release of Koha is not “uptodate” [sic]. Finally, Liblime has recently released a clients-only version of Koha. Though legal, this move raises serious questions about their commitment to the principles of open source software (Hadro, 2009). For these reasons, we chose to use the ByWater version of Koha in this analysis.
3. Observations and Findings
Below is a detailed discussion of our observations and findings for each functional area.
3.1 Patron Maintenance
3.1.1 Searching for Patrons
The patron search function is easy to use in both systems. Both handle partial inputs in searchable fields. However, where Evergreen uses ‘begins with’ searching, Koha uses a ‘contains’ and retrieves partial matches. In Evergreen, it is also easier for the user to see what fields are being searched and allows the user to use a single field or a combination of fields(see Figure 1). In comparison, according to the message on the search interface, Koha searches patron card number and name fields. This is consistent with the 3.2 documentation (as of March 12, 2011, and not mentioned in the 3.0 documentation). However, partial email matches also produce results (see Figure 2). This is not made transparent, and it is unclear to the user what fields are being searched as the search in actuality includes not only email, but also other names and user ID (Cormack, 2011).
Both systems display results with the most basic patron information, and allow sorting of results. Evergreen also allows customization of the results display and easy access to further patron details while still in the search screen (see Figure 3). It provides a button to return to the search form with the existing search still filled in. In comparison, Koha only has basic patron information display, a non-customizable results list, and no option to return to the search results (although the browser’s back button works).
At first glance, the only feature unique to Koha is the option to browse for patrons by last name. However, as Evergreen searches fields using ‘begins with’, to produce a result similar to that provided by Koha’s ‘browse by last name’ feature, the user would enter, for example, ‘A’ in the last name field, whereby Evergreen will list all patrons with a last name starting with ‘A’.
3.1.2 Adding Patrons
Adding patrons is fairly easy and straightforward. Both systems have the function available either directly from the main menu (see Figure 5) or from the main ‘Patron’ screen. The major difference here is that Koha forces the user to choose the user group to which the patron will be added. This might appear beneficial; however, the list of user groups may become unwieldy if it becomes very long, particularly because they are not automatically grouped when necessary.
|
Evergreen Sections and Fields |
Koha Sections and Fields |
||
| User Identification | Barcode Username Password Names (First, Middle, Last, Suffix, Alias) Date of Birth Juvenile? Primary Identification |
OPAC Login | Username Password |
| Patron Identity | Salutation Names (Surname, First, Initials, Other) Date of Birth; Sex |
||
| Additional Attributes and Identifiers | Driver’s License Favourite ColourPrevious Systems ID |
||
| Surveys | Customized by admin | ||
| Contact Info | Email Address Phone (Day, Evening, Cell) Home Library |
Contact | Phone (home, work, cell) Email (home, work) Fax |
| Alternate Contact | Names, Address, and Phone | ||
| Addresses | add as many as needed | Main Address | 1 address |
| Alternate Address | 1 address | ||
| Groups and Permissions | Profile GroupAccount Expiration Date Internet Access Level Active? Barred? Set as Family Account? Claims Returned Count Alert Message |
Library Management | Card Number Library Category; Sort 1, 2 |
| Library Set-up | Registration Date Expiration Date OPAC note Circulation note |
||
| Statistical Categories | Customized by admin | Patron Account Flags | Gone no Address? Debarred? Lost Card? |
| Finish | View Summary Save User; Clone User Cancel |
End Options | Save Cancel |
| Messaging Preferences | Hold Notification (set elsewhere) | Patron Messaging Preferences | Item Checkout; DUE; Check-in Hold Filled; Advance Notice |
Figure 4 – Patron Record Sections and Fields Comparison
The form itself is structured differently in the two systems. Whereas Koha has one long form, Evergreen has broken fields into sections and requires the user to select a specific section before entering the information (see Figure 5). The forms are organized a little differently as well. Koha separates the patron names, contact information, and address from their library identification (e.g. barcode number). Evergreen combines name and barcode into a single section for user identification, storing contact information and addresses separately. Other parts of the patron record also vary with the system (see Figure 4). While Koha allows more customization for general messaging (Evergreen only allows customization of hold notices only), Evergreen provides more flexibility by allowing more than two addresses. It is difficult to make a categorical statement about which is better, as it greatly depends on the needs of the organization. A nice feature of the Evergreen form is that required fields are not only marked red, but also automatically turn white when valid text is entered into a field (see Figure 5).
One very useful function that both systems have is notification of possible duplicate records based on first and last names. However, when a possible duplicate record is identified in Koha, a bug allows the patron record to be added without a card number (bug reported).
Additional features:
Evergreen
|
Koha
|
General error handling in the patron form for the two systems is fairly similar. If the user tries to save without filling in all the required fields or entering valid information, a fairly standard error message appears. Both systems’ error messages list all the required or invalid fields (see Figure 6). They serve their purpose, but could use some improvement. One simple improvement would be to provide a definition of a valid entry (which Evergreen does to some extent, see Figure 6); for example, telling the user that a date must be in the format MM/DD/YYYY, or a name field must only use the characters A-Z and a-z.
Field validation also seems minimal. Koha allows alphanumeric and symbol entry in the name field, which seems unusual. It may not have any validation beyond requiring an entry. Evergreen, in contrast, only permits letters in name fields (see Figure 5). Both have format validation for dates, but does not restrict the range of the date, allowing card expiration date to be earlier than registration/current date, and future birth dates. In Evergreen, an error message does pop up when first entered and after saving if the birth date is invalid or a future date, but the user can still re-enter a future date and save it (see Figure 5). No authority lists seem to exist in either system as any text can be entered into ‘Country’ for example. As a consequence, more advanced validation is also not present, such as no postal code format validation based on country. In short, error messages and field validation could be improved in both systems, but particularly in Koha.
3.1.3 Updating Patron Records
Evergreen is more complicated than Koha when the user wants to edit a patron’s information. The patron must first be found through the search, their information retrieved, and ‘Edit’ clicked on. The Koha interface has an edit option directly next to each patron in its search results lists. Nevertheless, both systems provide good consistency by providing the same form used when adding a new patron. There is one difference: when viewing patron details in Koha, the user can choose to edit only one section of the patron record. This is handy for the user, as this prevents the user from having to scroll through a long form. However, since these small sections are only accessible through the patron details, whether it is an increased convenience is debatable.
3.1.4 PIN Resets
PIN resets are slightly unintuitive and problematic in Koha. Although a staff member can manually change a PIN when editing a patron’s records, random password generation is available only by clicking the ‘Change Password’ in the patron details view. Koha also has no function for the patron to reset their own password, and there is no evidence that this function has been implemented even in the latest (3.2) version. In comparison, Evergreen has a ‘Reset’ button next to the password field when editing a patron record, and the OPAC has a “Forgot your password” link with a short pop-up form that will then send an email to the patron.
3.2 Check-in/out
The check-in/check-out module is highly accessible and supported by ample documentation in both Evergreen and Koha. It is an option on the default home page for both systems, although its position at the top centre of the screen makes it somewhat more visible in Koha.
The systems follow the same basic sequence of steps for circulating items: 1) identify patron; 2) identify item; 3) check item in/out; 4) print receipt (optional). While Koha allows you to search for patrons by name, email or barcode only, Evergreen allows you to enter a phone number or postal code as well. However, in Evergreen, a different keyboard shortcut is used depending on whether you are searching by barcode or by last name. The comparative simplicity of Koha’s interface makes it more user friendly in this area.
Checking an item out requires entering its barcode. At this point, the user also has the option of manually overriding the standard due date. In Evergreen, the date must be entered in YYYY/MM/DD format. Unless it is manually reset after checking out the item, this custom due date is applied to all subsequent items in the patron’s check-out session.
Koha’s custom due date feature appears more user friendly at first – it includes a drop-down calendar, for instance. A check box in the custom date area is labelled ‘Remember for session’, implying that settings will default to standard if it is left unchecked. However, here there is a glaring error: unless changed back manually, the custom due date remains in effect for the duration of the session, even if this box is left unchecked. Koha 3.0 also does not require any validation for setting custom due dates (see Figure 7, which shows a due date well in the past). This is clearly a problem, though it is fixed in Release 3.2.2. (Nighswonger, 2010).
In both systems, warnings and/or error messages pop up when the user attempts to check out an item to a patron who has fines or has exceeded the maximum number of items on loan. The specifics of these settings can be adjusted in the Administration area.
Since libraries accept book returns even when closed, an important feature of circulation modules is the ability to ‘roll back’ the due date of an item as it is checked in. This function is provided by both Evergreen and Koha, but is more straightforward in the latter system. Koha’s ‘dropbox mode’ can be programmed to automatically roll back the date according to a preset pattern – it ‘knows’ when the library was last open. In Evergreen, the user must enter the date manually.
The overall functionality of the two systems is very comparable for the check-in/out module, though Evergreen offers better support for federated libraries. Koha’s ‘Independent Branches’ function allows for basic sharing of material and patrons to be turned on or off, but is less flexible than Evergreen and does not have more nuanced separation features: either every branch set its own policy, or all branches share a single group of settings. In contrast, Evergreen offers finely granulated levels of access. However, Koha’s well-designed interface makes it a better choice here unless users are working at a large, federated library system with multiple branches.
3.3 Renewing Items
Both Koha and Evergreen support consortia in this area, allowing renewal policies to be set on local and on group levels. Also common to both are administrative options to allow staff override of renewals and custom renewal dates. Setting a custom renewal date requires two steps in Evergreen: first renewing the item, then extending the due date. It is possibly to simply extend the due date, but this will not count towards item renewals. Another minor inconvenience is the need to refresh the screen after renewing; the new date doesn’t appear automatically. The number of renewals allowed depends on the item’s Circulation Modifier, which is controlled under in the ‘Local Administration > Circulation Policies’ area. In the test case, the default circulation policy was recently adjusted to allow two renewals for items that do not specify their Circulation Modifier in the ‘Item Details’ area as for many users, it was previously causing problems by not allowing any.
The Koha staff client suffers from slightly more serious usability issues. The renewal function can be found in both the Details and the Check Out sections of the ‘Patron’ screen. In the ‘Check Out’ section, the function appears in multiple places (see Figure 8). The ‘select all | none’ options underneath the ‘Renew’ heading on the ‘Patron’ screen do not appear to do anything –also notice that this column does not contain any check boxes. Furthermore, the dual function of the ‘Renew or Return checked items’ is not ideal from a usability standpoint. During testing, items were appearing as ‘Not Renewable’, but it turns out the culprit was a bug in the circulation modifier rules. This option could be overridden using the check box provided (see Figure 8). However, after renewing, the status of the item reverts to ‘Not Renewable’. The user might therefore easily conclude that the renewal had been unsuccessful. The system would be improved if it provided better user feedback and more informative error messages. Error messages and alerts can be customized in the administration area, and the frustration encountered in this instance shows the importance of doing so. Finally, it would also be useful if Koha offered the option of renewing items from the ‘Item’ screen.
Renewing from the patron account is a more user-friendly option. In Koha, renewal status and renewal options appear beside checked out items in the patron account area (assuming that online renewal is turned on, which is an administrative option). Renewal status is accompanied by any necessary explanation, for example: ‘Not renewable (on-hold)’ (this particular setting, incidentally, can be adjusted in the administration area).
In Evergreen, the patron renewal interface is very similar to that found in the staff client. However, custom due dates are not an option, which simplifies things. The screen displays the number of renewals remaining, which is more helpful information than the number of renewals used, which is what Koha records. User reports do reveal one glitch: if a hold has been placed on an item, renewing that item will then be disallowed even if the hold in question has been cancelled, unless the item has been checked back in. This is an important point to be aware of.
3.4 Bills, Fines & Payment
Both Koha and Evergreen offer an extensive range of options for billing patrons and accepting payments. Please see Figure 9 (next page) for a detailed comparative summary of features.
|
Evergreen |
Koha |
|||
| Pros | Cons | Pros | Cons | |
| Overdue fines |
|
|
|
|
| Adding, deleting, and editing bills |
|
|
|
|
| Paying bills and fines |
|
|
|
|
| Accessing account history | History of fines, payments, and current balance accessible through OPAC and staff client | As mentioned, if bill for a lost item is voided, the item disappears completely from the patron’s account | Patron can view outstanding charges and payments in OPAC | No payment history available in staff client |
Figure 9 – Summary of bills, fines, and payment features in Koha and Evergreen
3.5 Holds
3.5.1 Placing/Editing Holds
Adding a hold is easy to do in both systems. Evergreen has an option from the main menu, and Koha has the option from the search results page. Koha has a convenient list of existing holds when placing a hold for a patron, whereas in Evergreen, the user must find the option from the ‘Actions’ menu. Nevertheless, Evergreen as greater flexibility than Koha when placing a hold at specific level (i.e. meta-record, title, volume, copy, see Figure 10). Though this function is not immediately obvious in Evergreen, it is well documented and can be learned quickly. Evergreen hasthe problem in that a held item will show as “Available” until the staff changes the status, and in Koha, this happens some of the time (known bug).
One feature both systems lacked was listing the number of total holds on an item, or the hold place of a patron. Without this feature, patrons may become frustrated if they have to wait a long time and may frequently take up staff time by inquiring about their holds. [I’ve been told this is in both, but it’s not obvious.]
Additional features:
Evergreen
|
Koha
|
3.5.2 Holds Pull Lists
Evergreen
|
Koha
|
Both systems are fairly with nice additions such as sortable fields. However, Evergreen seems to have no function to report a list of currently unfulfilled holds (and none of the available documentation suggests this function is in the new version).
3.5.3 Holds Capture
Evergreen
|
Koha
|
Hold capturing is definitely much better in Koha, and integrates well with its other functions. In Evergreen, when a patron puts a book on hold, it is not automatically marked as such and an item can be checked out by another patron if the patron takes an item off of the shelf before a staff member pulls it. Although the details are not clear, based on the documentation (2010), Evergreen also seems to send a notice to the patron when the item they placed on hold is checked in, and not when the item is ready for pickup. Clearly, Evergreen can use some improvement, particularly inintegrating their holds function with the rest of the circulation module.
3.6 Changing Status of Items
Evergreen
|
Koha
|
Although the two systems have similar functions, they handle them differently, presumably according to how the functions are integrated with the rest of the system. In both systems, item status can be changed through item search or through patron records. However, whereas Koha always takes the user to the item screen (and fines are calculated automatically based on its previous status), Evergreen gives the user different options depending the view. While Koha gives the user a more consistent interaction, Evergreen is more dynamic. Whether one is better than the other comes down to personal preference in this case. Nevertheless, Koha’s consistency may allow the novice user to learn the system more quickly. Evergreen, with its need for an item to be checked in before its ‘Lost’ status can be removed, is less intuitive.
Koha gives the user more options when marking an item as ‘Lost’, but Evergreen has the added benefit of having a ‘Claim Returned’ status for the patron. Workarounds are possible in Koha, but would need to be noted manually, which would not allow for automatic reports or statistics of the number of items that are claimed to be returned. However, Koha automatically calculates refunds when lost items are returned, which saves time for user and patron.
3.7 Changing Load Period
Changing the loan period, for example, of DVDs for juvenile/young adult patrons is very easy in Koha. A table in the section ‘Administration > Circulation and Fine Rules’ allows loan periods to be modified according to library, patron type, and item type (see Figure 11). In this case, the user would select the patron category ‘Child’ or ‘Young Adult’ and the item type ‘DVD’. Such a rule might already exist; fortunately, it is also easy to change it. According to the interface instructions, ‘To modify a rule, create a new one with the same patron type and item type’. The new rule will replace the existing one.
Evergreen comes with a set of ‘Circulation Modifiers’. These are categories designed to “control circulation policies on specific groups of items” (Evergreen, 2010). One of the default modifiers is DVD. It is possible to set three different loan durations for a Circulation Modifier: ‘Short’, ‘Normal’, or ‘Long’. This feature is typically used to control the circulation of DVDs on the item level. However, it is also possible to link a loan duration to a particular patron group. The local administration options can be set to allow patrons in the juvenile/YA permission group to take out DVDs for the two-week period and adults for the one-week period for example.
If one had a section of DVDs that were marked specifically for juvenile/YA patrons, another option would be to create a new Circulation Modifier to accommodate this. This too would be done in the administrative section. As in Koha, this can be done on the branch level or across the system. However, Evergreen has more advanced options for consortia; it allows clusters of branches to share settings, whereas Koha either applies one setting to all branches or requires each branch to set a policy independently.
4. Documentation & Help
Documentation for Koha 3.0 is not complete and seems to focus mostly on back-end settings. As the documentation is in a single PDF file, it is not easy to skim through, a little difficult to navigate without a linked table of contents, and has image markings in the wrong places. Novice users may be daunted by the casual use of technical terms and concepts. The Koha 3.2 documentation is much better in comparison. It is more complete, includes screenshots,and appears to be designed more for the end-user. This difference may be attributed to the fact that the 3.0 documentation was a student project, whereas the 3.2 documentation is a continuing community effort. Although Koha 3.2 differs from 3.0 in a few areas, we found its documentation material very helpful in our evaluation and frequently referred to it.
The Evergreen 1.6 documentation has fewer images and screenshots, but provides many step-by-step instructions and explains its terms when appropriate. On the whole, the text is better organized and easier to navigate, making good use of numbered and bulleted lists.
Informal support communities, such as listservs and wikis, are readily available for both systems. However, the schism in the Koha community has the potential to cause some confusion among users. LibLime refers users to a new support site, Koha.org, while the older communities are dominated by ByWater users. There is considerable overlap in their coverage, but they are discussing different systems.
5. Conclusions
Koha’s interface is friendlier and more streamlined than that of Evergreen. Modules integrate well with each other, as seen with automatic fine refunds and holds capture. It is generally more intuitive for users, even considering its sub-standard documentation for the 3.0 version since the 3.2 documentation can provide some support for users of earlier versions.As the staff client is web-based, there is also less maintenance needed. The OPAC also has the added features of allowing patron tagging, comments, and reviews. Müller (2011) also did an evaluation of several free and open source software ILS based on software licensing, community, and functionalities for large libraries, and found that Koha was the leader in all three categories and was the only to pass the full evaluation. Nevertheless, Müller advises that Evergreen should also be seriously considered, which may be all the more true with the recent release of Evergreen 2.0.
Evergreen provides more flexibility and functionality for consortia (see Figure12). In addition, our analysis above found a number of features that, though available in both systems, were designed better in Evergreen (PIN resets, for example). Furthermore, its documentation is superior to that of Koha.
| Consortial Features | Evergreen 2.0 | Koha 3.2 |
| Library Groups | yes | yes |
| Settings for Groups | yes | |
| Multiple Branch Managements | yes | yes |
| Floating Collection Management | yes | in development |
| Granular User Permissions (per library) | yes |
Figure12 -Consortial features in Koha and Evergreen (RSCEL & OS-OL, 2010)
Overall, the systems are very comparable. Libraries that belong to consortia may decide that Evergreen is the better option for them; the advantages that it offers in this area will warrant its steeper learning curve. Special libraries and stand-alone branches, on the other hand, might prefer to opt for Koha as it generally has a more intuitive user interface. Ultimately, as is always the case when selecting software, the better choice depends on the needs and requirements of the organization.
Works Cited
Breeding, M. (2009). Chapter 3: Major open source ILS products. Library Technology Guides, 44(8), 16-31. Retrieved March 12, 2011 from http://alatechsource.metapress.com/content/t75710j302n13447/
Breeding, M. (2011, January 27). Perceptions 2010: An international survey of library automation. Library Technology Guides. Retrieved February 19, 2011 fromhttp://www.librarytechnology.org/perceptions2010.pl
Cormack, C. (2011, March 12). Question on Patron Search. Message posted to Koha electronic mailing list, archived at http://lists.katipo.co.nz/pipermail/koha/2011-March/028119.html
Evergreen Community.(2010). Evergreen 1.6 documentation. Retrieved March 12, 2011 from http://docs.evergreen-ils.org/1.6/draft/html/
Hadro, J. (2009, September 22). Liblime’s Enterprise Koha sets off debate. Library Journal.Retrieved February 19, 2011 fromhttp://www.schoollibraryjournal.com/lj/technologyproductsvendors/855795-296/story.csp
Hellman, E. (2010, January 29). Who owns Koha? Retrieved February 17, 2011 from http://go-to-hellman.blogspot.com/2010/01/who-owns-koha.html
Koha Library Software Community.(2011). Documentation. Retrieved March 12, 2011 from http://koha-community.org/documentation/
Leonard, O. (2010). The Nelsonville Public Library chooses ByWater Solutions. Koha Newsletter, 1(2). Retrieved February 17, 2011 from http://koha-community.org/koha-newsletter-volume-1issue-2-february-2010/
Müller, T. (2011). How to choose an free and open source integrated library system. OCLC Systems & Services: International digital library perspectives, 27(1), 57-78. Retrieved March 12, 2011 from http://eprints.rclis.org/bitstream/10760/15387/1/How to choose an open source ILS.pdf
Nighswonger, C. (2010, December 2). Koha 3.2.2 is now available. Retrieved from http://koha-community.org/koha-3-2-2/
RSCEL & OS-OL.(2010). OSLS Feature Comparison Matrix.Open Source Open Libraries. Retrieved March 12, 2011 from http://www.os-ol.org/features