Cynthia A. Sikora
Lucent Technologies, Bell Laboratories
Holmdel, New Jersey
Linda A. Roberts
BellSouth
Atlanta, Georgia
Bonnie M. Kudrick
Titan Corporation
Mays Landing, New Jersey
In this research, we investigated characteristics of web searches for specific topics as well as general trends across topics. Screen activity data captured from web searches conducted on four search topics by 15 subjects in a study by Rieh and Belkin (2000) was analyzed. The search topics investigated in detail included Research Interest, Medical Disease and Conference Travel. Although the number of web sites visited was not significantly different across tasks, the number of pages visited was significantly different. The results indicate that the increase in revisiting pages in some tasks was not associated with an increased use of the back button. Although the breadth of the search structure representations did not vary considerably among search topics, the depth varied substantially. Searching for information on a medical disease resulted in a much more predictable and limited search structure compared to the other search topics. The common trends and task-specific characteristics identified provide guidance for addressing users’ needs when designing information systems.
INTRODUCTION
It is important that Information Systems support users’ tasks and their style of searching. Identifying trends in web searching that are constant across search task helps guide system design decisions. It is also important to understand the task-specific behaviors that users adopt. These changes
in search approach based on search topic should dictate corresponding features or behaviors in the information retrieval system that support the users. The specific task and the devices used to access the web can change the parameters that optimize the use of the site. Parush and Yuviler-Gavish (2004) explored the appropriate hierarchy depth of web sites for optimal navigation using small screen devices. Measurements of navigation times and success rates provided evidence that broad navigation structures were superior for web searching on small screens as well as typical computer screens. The
authors further suggest that deep hierarchical structures provide the disadvantage of increased working memory load, confusion and disorientation. This data would suggest that the content of web sites be structured to disuade users from continuing down a path to deeper and deeper levels to obtain information. However, if a deeper hierarchical structure is necessary based on the task, systems will need to find ways to support the users in the deep structure. In addition to forward navigation, supporting users’ ability to easily find their way back to where they have been is
critical.
Chi, Pirolli, Chen and Pitkow (2001) observed that searching for information on the web by traversing links on web pages was a common trend for users. This page to page movement allows users to move from one web site to another or to move to deeper levels of detail within one website. Understanding the complex interactions between user needs, user actions and the structure of the web content as it is searched is both theoretically interesting and practical in its implications.
Katz & Byrne (2003) investigated web site navigation using a menu of links versus the web site search function. They found a wide distribution of searching and browsing behavior across sites and users. They also found that browsing behavior was influenced by the breadth of the menus. This provides evidence that users’ search styles vary based on circumstances beyond individual differences. This study investigates the common and contrasting characteristics of web searches on different search topics. To the extent that the same set of users demonstrate differing searching behaviors with different search topics, there will be additional evidence that users change their search style to address varying tasks. Common characteristics across tasks will provide guidance as to features or behaviors that may need to be supported uniformly regardless of the search task. An additional goal of this investigation is to identify a search task with a limited hierarchical depth and breadth to be used for controlled studies in the future. A search task that results in a limited and predictable search structure will allow research to be conducted on searching while mapping feedback elements to the search structure. A search structure that is too broad or too deep may extend beyond the feedback elements built into a testing simulation.
Therefore, identifying a search task with a predictable search structure will facilitate subsequent studies on multimodal cues in web navigation.
METHODOLOGY
This study was designed to understand the characteristics of web searches that use differing tasks.
Specifically, a task was sought that would have predictable search patterns. Web search data from a study by Rieh et. al. (2000) was analyzed to determine trends in web searching and to identify a search task that could be used in subsequent studies.
Subjects
Fifteen subjects participated in the web search study. They included faculty members (N=6) and doctoral students (N=9) recruited from diverse disciplinary areas at Rutgers University. Faculty members were affiliated with the departments of communication, library and information science, linguistics, sociology, chemistry, and computer science. Doctoral students were in the departments of communication, library and information science, biomedical engineering, computer engineering, and organizational psychology. The average age was 32.2 for the doctoral students and 45.3 for the faculty. There were twice as many men (N=10) as women (N=5) conducting the web searches. All participants indicated a high degree of web experience and regular use (Rieh et. al., 2000).
Search Tasks
In a lab setting, each subject completed four generic search tasks that were related to the subjects’ research projects, travel, medicine, and computer prices. Subjects were advised to take 15 minutes to complete their search on each of the assigned search tasks. All screen activity of the 60 web searches was recorded using ScreenCam. This application records the web search activity with no noticeable degradation in response time and can simply be played back during the analysis phase. The search tasks assigned were intended to be diverse. They included:
1. Medical Disease – Search for an obscure disease called Schistosomiasis.
2. Conference Travel – Search to investigate travel and lodging arrangements for a
conference.
3. Research Interest – Research a topic of interest and acquire relevant information
4. Computer Purchase – Explore the best buy on a new computer system.
All searches began at the Rutgers University homepage to encourage subjects to choose their preferred search engines.
Data Analysis
The web search data recorded included 1321 web pages from 60 searches by the 15 subjects. Although the subjects had four search topics that they used as a basis for a web search, only three were analyzed for the purpose of this study. The three search tasks analyzed were:
1. Medical Disease
2. Conference Travel
3. Research Interest
The search structure of the Computer Purchase task was consistently narrow and very deep. It would not have been appropriate as a limited, predictable search task for use in future studies. For this reason, it was not investigated further. The ScreenCam data for each search was analyzed and then verified. Specific elements of the search were recorded including:
1. Search time
2. Number of pages visited
3. Unique sites visited
4. Search engines employed
5. Use of the back button
To determine the depth and breadth of the structure of the searches, each search was visually mapped out to assess the pattern of each search as well as to compare among searches. An example of the actual data recording can be seen in the mapping of a simple search in Figure 1. A simple search is defined as one that visits relatively few web pages. In this example, three search engines were accessed before the subject found a relevant link to pursue. The search representation shows a breadth of three and a
depth of eight, which would not be considered particularly broad or deep. A complex search would be characterized by visits to numerous web pages through an elaborate route and is illustrated in Figure 2. The breadth of this example can be determined from the four major branches while the depth of 13 is calculated from the Rutgers homepage down to the lowest level link. Although not a particularly broad
search, this example does represent a deep search.
Sample: Simple Search of Conference Task
Figure 1. A simple search is one that visits fewer Web pages. Complex Search of Conference Task
Figure 2. A complex search visits numerous Web pages through an elaborate route.
RESULTS
The subjects had a mean search time of just over 13 minutes (M = 13.57, SD = 3.26). Although subjects were provided guidance to spend 15 minutes on each search, the actual time spent varied somewhat. Overall, the average amount of time spent on a page was just over 30 seconds
(M = 35.50).
During web searches, subjects used approximately three different search engines on average (M = 2.72).
YAHOO! was the most frequently used search engine. Although some subjects never used the back button, on average subjects used the back button 25% of the time during the web searches. There was no significant difference in the frequency of using the back button in searches on the different topics, F(2,42) = 1.67, ns. The average number of unique sites visited was approximately four sites for every three minutes of search time (M = 18.22, SD = 7.03). Although there was no significant difference in the total number of unique web sites visited in each of the search tasks [F(2,42) = 1.95, ns], there was a significant difference between search topics in the number of pages visited [F(2,42) = 3.35, p<.05]. This suggests that the topic of the search was directing the subjects to visit pages differently. Pairwise comparisons indicate that the Research Interest search and the Medical Disease search were significantly different from each other in number of pages visited, but neither was significantly different in that dimension from the Conference Travel search. This can be seen in Figure 3.
31.73a
26.13ab
21.87b
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
Research
Interest
Conference
Travel
Medical
Disease
Search Topics
Number of Pages Visited
Figure 3. The mean number of pages visited is shown for each search topic.
An analysis of the visual representation of the search structure produced during the search task was conducted to assess how wide and deep the resulting structure became. The three search topics generated search structures that were fairly consistent in breadth, but varied in depth. The search structure was generally only a few nodes at its widest point. Typically, the widest point was the top of the hierarchy where the sites visited branched off from the search engine used. The depth that subjects went down a branch varied within a search topic. It was typical for some subjects to go down 15 levels. Although there was one subject in the Medical Disease task who went very deep in the search, that search topic tended to have fewer levels of depth in the search structures. This suggested that the Medical Disease search topic was more predictable than the Conference Travel or Research Interest searches.
DISCUSSION
The results indicate that many of the characteristics of the data generated from the different search topics are similar. However, the depth and breadth of the search structures varied based on the search topic.
The limited variability in time spent on the task is not surprising given that the subjects received guidance as to how long they should spend on the search topic. This relative consistency in the time spent on the task also contributed to the similarity in the number of sites visited during a search. However, in the similar time intervals with similar numbers of web sites visited, subjects searching on a research topic of interest moved between pages more than those looking for information on the medical disease schistosomiasis. Interestingly, the total number of pages visited was not a discriminating factor among any of these search tasks. This would suggest that returning to previously viewed pages is limited when looking for information on an unknown disease, more likely when making travel arrangements and most likely when investigating a research topic of interest. This finding becomes more interesting considering subjects did not vary significantly on their use of the back button in the different search tasks. Viewing previously accessed pages in the Research Interest searches may have resulted from relying more heavily on navigation links in the pages of the web site or through accidental stumbling upon the same page from different places. As a within subject design, the same subjects conducted the searches on each of the topics. Individual differences in search style would not explain the varied reliance on the back button unless there was an interaction with the way the individual approaches different search tasks. This may be an area of future investigation.
The search tasks were analyzed to determine the depth and breadth of a typical search for each of the topics. One of the objectives was to identify a search task with a predictable and limited search structure for use in a future study mapping feedback elements to nodes of the search structure. To develop a sufficient set of feedback stimuli for the potential search structure during a search simulation
study, it is necessary to identify the outside boundaries of the search structure in depth and breadth. The ideal search task would lead the subjects through multiple levels of links within and between sites, but restrict the depth and breadth of the search structure to a predictable number. The task
would also need to avoid having all the relevant sites accessible from the same higher-level site. Based on the data from this study, the Medical Disease search task had the most controlled search structure. The depth and breadth of the Medical Disease search task is shallow and narrow compared to the other search topics investigated.
If a different disease was selected for the Medical Disease task would a search structure with a controlled depth and breadth still have been demonstrated? It is likely that the Medical Disease search would have had similar results regardless of the disease in question, assuming it was still a little known disease. The subjects seemed to need to do much more exploring of different pages on multiple sites to get an overview of the disease. If the subject has familiarity with the disease before beginning the search, the resulting search structure may be quite different. It is reasonable to assume that the Research Interest search was diverse enough to produce results that are robust. Subjects in the study selected their own research interest for their search. It is unlikely that the study environment altered the search approach that they would have employed elsewhere. There was a tendency to find a useful path and drill down. Given that these are research topics that they have an interest in, they also probably have some familiarity with the topic. This would encourage the subjects to follow a relevant path of an appropriate site when a new facet of the topic was identified rather than to browse many sites. Moreover, tt is likely that the subjects were already familiar with the best sites to start their investigation of the topic.
The Conference Travel task had less inherent variability than the Research Interest task, but the subjects
selected conferences with varied locations and dates. The tendency for this search task to result in fairly deep search structures may have reflected the nature of the task. After comparing some initial price estimates, the subjects were likely to move down the path of the preferred travel web site to make all of the arrangements. An actual search for travel arrangements would likely result in similar search structure characteristics. The finding that the Research Interest and Conference Travel tasks yielded variable and often deep search structures are likely representative of the results that would be found conducting those types of searches outside of the lab environment. Understanding the commonalities and differences in web searches based on the specific search topic can provide insight into how best to create systems to support the users’ tasks. This data would encourage system designers to support easy access to multiple search engines as well as the ability to go back to the previous page for all types of searches. It would also suggest that systems should support the ability to use multiple exploration methods to approach different topics. The evidence that users often return to previously accessed pages without using the back button suggests that history mechanisms might be useful.
REFERENCES
Chi, E. H., Pirolli, P., Chen, K., & Pitkow, J. (2001). Using information scent to model user information needs and actions and the Web. Proceedings of the SIGCHI conference on Human factors in computing systems, 490-497.
Katz, M. A. & Byrne, M. D. (2003). Effects of scent and breadth on use of site-specific search on e-commerce Web sites. Proceedings of ACM Transactions on
Computer-Human Interaction, 10(3), 198-220.
Parush, A. & Yuviler-Gavish, N. (2004). Web navigation structures in cellular phones: the depth/breadth tradeoff issue. Proceedings of the International Journal of
Human-Computer Studies, 60, 753-770.
Rieh, S. Y. & Belkin, N. J. (2000). Interaction on the Web: Scholars’ Judgment of Information Quality and Cognitive Authority. Proceedings of the Annual Meeting of the American Society for Information
Science, 37, pp. 25-38.
