Research Abstracts - 2007

Motivation

As the Web has grown and matured, new technologies such as scripting languages, cookies, and XML have been introduced that have made the Web much more dynamic and personalized than before. At the same time, Web interaction has become more complicated and involved [1]. Users are often required to correctly remember and input user name and password combinations and fill out sequences of tedious forms to perform even relatively simple tasks. Today, web pages are often dynamically generated, and so it is not always possible to use traditional bookmark facilities to retrieve them. This is unfortunate, because it is precisely these kinds of pages that require the most effort from users to repeatedly access, and which would therefore benefit the most from a bookmarking facility that allows dynamic content to be retrieved with a single click, as traditional bookmarks provide for static web pages today [2].

There may also be instances where users find themselves repeatedly performing the same tedious sequence (or similar sequence) of tasks. In these cases, it may be very beneficial to users if they could easily automate and save the task sequence they are repeating so that by entering just one or two parameters each time (the departure and return dates for a flight search for instance), they can retrieve the information they want much more quickly.

A potential third benefit of this kind of dynamic bookmarking capability is that it would allow users to easily share dynamically generated content. For example, a user could customize a computer he or she wishes to purchase online, save the completed configuration page as a bookmark, and send the bookmark to a friend to look over.

Approach

Smart Bookmarks has been implemented as a sidebar inside the Firefox web browser. From the Smart Bookmarks sidebar, users can bookmark their current web page, run a bookmark they have already created, or view and edit a graphical representation of an existing bookmark. Smart Bookmarks continuously monitors and records the actions the user makes as he or she browses the Web and stores them as commands in the background. Then, when a web page is bookmarked, it automatically determines the necessary sequence of commands from the user’s browsing history needed to return to that page and saves those commands as a bookmark.

Figure 1. Smart Bookmarks sidebar inside Firefox

Once a bookmark has been created, it can be run with a single click. Smart Bookmarks will replay the bookmark’s command sequence in the user’s web browser to return to the page that was originally bookmarked. Users can also indicate that they should be asked about the inputs for certain commands in the bookmark before each run. This would be useful for automating a task where only one or two things change each time the task is performed.

In order to clearly show users what a particular bookmark will do when run, Smart Bookmarks displays a graphical representation of the bookmark. The graphical view includes a screenshot of each command in the bookmark showing the part of the web page that command acted on when originally created. Smart Bookmarks also allows users to easily edit a bookmark once it has been created, for instance by changing the inputs to a command, copying commands, or removing a command from the bookmark entirely. Users can dynamically edit or even create bookmarks in real time by explicitly performing the actions they want to capture in their browser.

Figure 2. Graphical view of a bookmark

Current Work

One issue with automatically capturing browsing actions and making it easy for users to replay those commands later is that it is possible that one or more of those commands may have some undesirable side effect that the user does not want to be repeated. For instance, if the user purchases a book online or requests to join a mailing list, he or she probably does not want to repeat those actions when running a bookmark. While the graphical bookmark representations should make it easier for users to catch when cases like this may occur, a more reliable and less error-prone solution is desired. To that end, we are currently working on a two-part system that detects potential undesirable side effects when a bookmark is run by using various heuristics in combination with a shared database that will store votes provided by users of Smart Bookmarks about a particular command’s possible side effects.

References

[1] Anupa, V., Freire, J., Kumar, B., and Lieuwen, D. "Automating Web Navigation with the WebVCR." Proc. WWW9, May 2000.

[2] Safonov, A., Konstan, J., and Carlis, J. “Beyond Hard-to-Reach Pages: Interactive, Parametric Web Macros.” Proceedings of the 7th Conference on Human Factors & the Web, June 2001.

[3] Michael Bolin, Matthew Webber, Philip Rha, Tom Wilson, and Robert C. Miller. "Automation and Customization of Rendered Web Pages." UIST, 2005, pp. 163-172.