Facet Catalogs DL
Beyond Text Querying and Ranking List: How People are Searching through Faceted Catalogs in Two Library Environments
Xi Niu 2010
LOG FILE (No investigation)
Facets in Library Catalogs: The Beliefs, Behaviors, Policies and Practices That Guide Implementation
A Thesis
Catherine Elizabeth Hall 2016
Study of User Search Activities with Two Discovery Tools at an Academic Library
(investigate and compare user search activities of two discovery tools at an academic library.)
Xi Niu 2014
As libraries have introduced facet-based catalogs, a growing body of literature has resulted which reports on user experience with facets. These studies usually employ small sample sizes, and can lack rigorous methodologies and statistical testing; however, they do provide valuable information on user interaction with real-world systems containing large and complex datasets (Fagan, 2010).
Hearst, 2006 Researchers at UC Berkeley, led by Marti Hearst, were at the forefront of developing facet-based interfaces for digital collections, building a system, Flamenco, to determine how to aid navigation and browsing of collections via the use of hierarchical faceted metadata
GAP:
Furthermore, as Thomsett-Scott and Reese (2012) pointed out, these user studies of discovery tools were mainly for system testing and validation purposes. It is thus more observe users search behavior in action, in order to obtain further valuable information regarding the impact of discovery tools.important to assess and
Several studies (Becher & Schmidt, 2011; Denton & Coysh, 2011; Williams & Foster, 2011) reported that users prefer facets for refining search results and distinguish between types of materials, but it is not clear to what extent users use facets in their search process.
Recent literature on discovery tools have been focused on usability and user acceptance (Comeaux, 2012; Denton & Coysh, 2011; Emmanuel, 2011; Williams & Foster, 2011),discussions on system design and implementation (Daniels & Roth, 2012; Wrosch, Rogers-Collins, Barnes, & Marino, 2012), information literacy and instruction (Buck & Mellinger, 2011; Fawley & Krysak, 2012), and impact on library collection usage (Way, 2010)
Hofmann and Yang (2012) provided many up-to- date facts about discovery tools used in academic libraries
As the additional coverage of electronic resources introduced new search options, facets, and search results display, there is a need to investigate whether users alter their search tactics when they are searching for electronic resources compared to the traditional catalog items such as books and print materials.
Additionally, faceted search has become a standard approach for academic libraries to provide information access for users. Since both VuFind and Primo support faceted search and browsing as one of the key features, it is important to examine and compare users’ facet selections with the two discovery tools to see whether the coverage difference would lead to different facet usage.
TASK ⚠
The literature on OPAC studies suggests that people primarily conduct two types of searches using OPACs (Hancock-Beaulieu, 1990). One is the known-item search where the user wants to find a specific item using information such as author, title, and publication year. In contrast, another type of search frequently conducted by users is the subject search, which is conducted on a topic using either a keyword or a subject heading.
Known-item searches and subject searches can also be called close-ended and open-ended searches, respectively, because the former has a definite target document and the latter has more open-ended target documents.
This finding confirms conclusions published from previous studies that most people started with the broadest and default search, that is, keyword search (Lown, 2008; Niu & Hemminger, 2011; Pennell & Sexton, 2010).
Compared to keyword searches, other fields were complementary and supplemental, and used only in a smaller number of search sessions
One most likely reason for this higher percentage is that two drop-down menus (format and exact phrase search) next to the search box in the Primo interface might be distracting for users. When facing a possible choice overload in a search interface, users may respond by keeping every setting as default. Another possible reason might be that Primo is intended as a “one-stop” search for all library resources. Users like to apply minimum search effort with Primo as they would when using search engines like Google and Bing.
Topic (or Subject) is another frequently used facet in both VuFind and Primo. Topic is content-related and it is based on the Library of Congress Subject Headings (LCSHs), which may be difficult to understand for users without some training or knowledge. The relatively high usage of Topic (or Subject) facet suggests that the patrons were able to take advantage of the authority data to access the library’s collections. Although some facets are frequently used, it is difficult to find a highly used value under that particular facet. Users used a variety of facet values with each value has been used only once or twice. For example, facets like Topic and Author do not have any particular popular values. As a whole, these facet values are collectively helpful for users but there is no single frequently selected value, due to the unlimited enumerative nature of these facets.
Query Formulation/Reformulation
compares the average query length (the number of words used in a query)
the percentage of the searches that were reformulated
the number of query submissions per search session
users may be more efficient in searching electronic resources because they performed fewer search
iterations
The fewer search iterations for electronic resources suggest that users were able to find electronic materials quickly in Primo and thus did not additional iterations of search query.
The percentages of the searches that were reformulated are about the same for non-electronic resources (61.0%) and electronic (57.8%). That means less than half of the searches in Primo (i.e., 42.2% for electronic and 39.0% for non-electronic resources) had only one query submission. These percentages are roughly consistent with previous studies. For example, Spink et al. (2001) concluded that around half of users (52% of the users in the 1997 Excite dataset and 45% of the users in the 2001 Excite dataset) reformulated or made modifications to their initial queries.
As to the query reformulation, three reformulation strategies are identified from the log data: narrowing, parallel, and broadening
More users tended to narrow a search than to broaden one. Narrowed queries are typically longer than the original ones and are assumed to lead to higher satisfaction (Belkin et al., 2003)
Users narrowed down most searches by adding one or several terms to append some specific information, such as content, time, or format. Examples of narrowing searches are:
Parallel movement of searches involves synonym replacement, format change, and spelling correction. Some examples are:
In general, many of the queries beyond the first iteration were simple deviations from the initial one. Some the query modifications were were performed to correct typographical errors. This observation is in line with White and Marchionini’s (2007) finding that many further queries were simply “syntactic variants” of the initial one. Therefore, the initial query is very important in determining search success.
Compared to narrowing or paralleling movements, broadening activities were much less common for patrons. Most broadened queries were the shortened version of the previous queries and were created by removing one
If users modified their original queries more than once, they rarely persisted in narrowing down or broadening up through successive trials. Most users would use mixed strategies of narrowing, broadening, and parallel
48 tasks performed /eight participants
open-ended tasks that required participants to find recent journal articles in the area of supply chain management.
Success of Search Task Performed. need to be measured
FINDINGS (1) keyword search was dominant in text search for both tools; (2) faceted actions were less common compared to text search; (3) most search sessions were very brief with only a few actions (less than four query submissions) and the queries users typed into the search box were usually two- or three- term words (4) most search sessions (>50%) had the original queries reformulated.
Other limitations involving logs included the inability to determine searchers’ intentions, demographics, and satisfaction, could be complemented by the user study.
previous study simulated tasks that "may not resemble users’ actual situations"/ search task complexity by nature is fuzzy, and not rigorous enough to make the tasks “similar” for both discovery tools
The lack of a clear definition for task complexity has hindered the construction of the topics due to the lack of guidance criteria in the field
why it should be naturalistic? Participants varied in their interpretations of the topics, and some of them had previous knowledge that made them perceive a task to be easy. In terms of task efficacy, the degree to which tasks depend on the interface, and to what degree they depend on individual differences, is difficult to discern. With hindsight, the tasks were controlled at the aggregate level.
As libraries are implementing new discovery tools, the integrated approach we developed in this study involving transaction log analysis and user testing could be extended to similar situations for assessing users’ search activities, in effect providing an empirical basis for selection of search options, facets, and search results presentation in discovery tools.
FACETS
The much-heralded attraction of most next-generation discovery software is the ability to provide facet-based search and browsing (Denholm et al. 2009). Facets can help reduce the frustrating experience of unhelpful results sets, where too many, too few, or even zero, results are returned (Antelman et al., 2006). Using facets as post-coordinate refinements can reduce a large result set to a manageable size and, since facets and their content is only visible if there are information objects associated with them, should not allow a user to follow a path that leads to zero results. Library and information systems have traditionally been designed to support search queries, and few allow for a true browsing experience. Facets may assist with browsing by providing the scaffolding to allow information seekers the opportunity to explore previously unknown or unrecognized avenues within the catalog. Facets also allow users to move through large information spaces in a flexible manner without feeling lost (English et al. 2002). When an initial search returns an overwhelming number of results, or an information seeker is struggling to frame their information need, facets can help with the selection of pertinent avenues of enquiry, and siphon off irrelevant results.
Since 2006, the number of libraries offering facet-based catalogs has increased at a rapid pace. In 2009, Hoffman & Yang (2011) randomly sampled 260 academic libraries in the United States and Canada and found that 16% offered facet-based catalogs. Revisiting the sample in 2011, they found this figure had almost doubled to 29% (Hoffman & Yang, 2012). Looking at 100 top academic institutions and 100 top public libraries, Hall (2011) found facet-based catalogs were offered by 78% of the academic libraries and 54% of public libraries.
However, data applied to cultural objects are more complex and powerful than most metadata that is routinely assigned to commercial products (La Barre, 2007) suggesting that facets must be chosen and implemented with considerable care to be of optimum use in a library catalog.
Thirty-two art history students participated in the evaluation study and were asked to complete four tasks on both interfaces. Ninety percent of participants preferred the faceted interface overall, and 97% said it helped them learn more about the collection (Yee et al, 2003).
the faceted interface outperformed the traditional interface on measures of ease of use, simplicity, flexibility, interest, ease of browsing, and enjoyment.
Throughout the study, the researchers also observed facet usage. With structured tasks, the use of facets tended to, unsurprisingly, be driven by the nature of the task and the most frequent starting points were date, location, media, artist, and theme. In unstructured browsing tasks however, facet usage was more evenly distributed across all facets. Within the browsing tasks, participants also used multiple facets simultaneously 19% of the time, and this rose to 45% of the time in the structured searching tasks (Yee et al, 2003).
Hutchinson, Druin and Bederson, 2007 children facet
As the first academic library to adopt a next-generation OPAC, NCSU planned a series of assessment activities to compare student performance using the old and new catalogs. The average task duration was significantly decreased with the new Endeca catalog. Although the number of failed tasks using Endeca was lower than with the old catalog, it was still reasonably high. Researchers concluded that this was partly due to confusing terminology in the new OPAC. (Antelman et al, 2006)
participants to find a certain book at a particular campus library. The Library facet was eventually used by most of the participants, but it was not immediately used by any (Littletree, 2008).
At the University of Minnesota, participants in user studies found facets to be useful for refining results and a differentiating feature between old and new catalogs (Sadeh, 2007b; Fagan, 2010). However, one year after the facet-based Primo catalog was launched as the default catalog, the library surveyed catalog users about perceptions of success, known- item searching and preferred search environments and found higher reported success rates with the traditional catalog than with the facet-based one (Hessel & Fransen, 2012).
The researchers caution that users of the traditional catalog are a self-selecting group who may be more likely in general to experience greater search success. They also note that a high proportion of known-item searching still occurs in the traditional catalog, whereas the facet-based catalog is often used for browsing topics (Hessel & Fransen, 2012).
The University of Kansas Library, another Primo user, performed user studies of its catalog shortly after its soft and full launches (Hanrath & Kottman, 2015). As is common in most catalog user studies, participants were presented with both known-item and open- ended research tasks. Across all tasks, facets were used 32% of the time and this figure rose to 55% for the open-ended task. Researchers compared their study findings to facet usage data collected by Google Analytics and found that use of facets in real information seeking activities was “not wildly dissimilar” (Hanrath & Kottman, 2015).
In the University of Chicago’s initial user study of its AquaBrowser catalog, nine of the twelve participants remarked that they found material in the new catalog that they had not previously discovered, and the researchers were confident that at least four of these
45
individuals identified new material through the use of facets (Olson, 2007)
Reaction to the facets was positive overall, with several participants finding them helpful as a refining tool. A small number of students were identified as being more ambivalent to the facets, at least for their own research, but stated that they might be more useful in supporting users less familiar with the library catalog (Olson, 2007).
York University Library conducted a usability study using both in-person testing and an online survey to assess its VuFind catalog (Denton and Coysh, 2011). The in-person usability test was based on previous studies conducted at NCSU and Yale (i.e. Antelman et al., 2006; Bauer 2008a, b, c) and ten students participated. The online survey was completed by 75 people. Participants expressed a preference for Vufind over the traditional library catalog and made particular mention of facets as the feature of the catalog that they liked and that stood out the most.
Similarly, all 16 international students at the University of Sheffield expressed a preference for a facet-based interface when interviewed (Tam, Cox & Bussey, 2009). They found facet interfaces to be familiar from commercial environments and felt that facets would save time, narrow results, and provide the functionality of advanced search while being easier to use.
Salaba and Zhang (2009) introduced 98 LIS students to library catalogs using AquaBrowser, Encore, VuFind and Scriblio systems and asked participants to rate features for desirability and helpfulness. Facet-based navigation was rated as a top feature for helpfulness alongside advanced search capability, sorting results, simple search box and enriched content. Facets were also judged to be the most desirable catalog feature, just ahead of relevance ranking.
A preference for facets over other features was also seen
46
at Western Michigan University where data analysis of the VuFind catalog found that approximately 20% of searches made use of facets but other web 2.0 “bells and whistles” were not used (Ho et al. 2008).
By reviewing user studies and usability tests and transaction logs of real-life use of facets, we can observe that while facets are generally praised they are not always used. Non-use of facets may have many causes, including the negligible benefits that using facets offers to many known-item or quick look-up style searches. In other cases however, it may be that non-use is caused by other factors such as uncertainty over what facets are or what they do, confusion about terminology, and unease about apparent inconsistencies and errors in the metadata.
Although the artificial search scenarios greatly affected the percentages, context may also contribute to the differences. (comparing ^ the log file & lab study outcome) Xi Niu 2015
The study is also limited by constraints imposed on the open-ended tasks. Only three target items were required for each open-ended task to measure the search time. This constraint reduced the exploratory nature of the open-ended tasks and made it difficult to capture the quality of the search in terms of recall, exploration paths, search diversity, and search depth.
Future research might consider removing the constraint on the open-ended tasks and investigating the subtle relation- ship between search time and search performance.