Abstracts - 2007
Getting to Know You Gradually: Personal Lifetime User Modeling (PLUM)
Max Van Kleek, Michael Bernstein, Robin Stewart, David E. Karger, MC Schraefel & Howard E. Shrobe
Progress in user modeling over recent years has demonstrated that models learned from observing users' actions can boost ease and efficiency of application use, improve interaction quality, and save users time and effort. Yet, despite progress in the field, relatively few applications on the desktop today employ user modeling techniques to adapt to users' needs. The field's most visible successes have instead been in recommender systems for online retailers and content providers, which gain leverage by simultaneously amassing profiles of hundreds, thousands, or millions of users. While this approach has been successful for online businesses and marketplaces, it is not easily applied to desktop applications, which have one primary user, and where information may be much more personal and sensitive in nature. One of the primary obstacles to user modeling on the desktop has been the complexity needed to develop application-specific user modeling systems to learn from user actions. Another is the bootstrapping problem, that very little about the user is known when the application is first installed on the user's system.
Our belief is that some of these desktop modeling challenges can be mitigated by decoupling user modeling components from applications, so that models can be shared across applications. In addition to reducing the bootstrapping problem, an advantage to this approach is that it becomes possible to capture task-related contextual connections among applications such as an e-mail client, web browser, and a text editor, which would otherwise be missed by application-centered modeling techniques.
This project seeks to advance techniques in personal lifetime user modeling (PLUM), the idea of building and incrementally refining models of a user through passive observations of their behavior, captured over a long term using their own digital devices. Our overall goals with this project are the following: first, 1) perform primary investigative work in PLUM, specifically to identify the primary major technical challenges and major risks for PLUM modeling approaches as a whole; 2) to establish standard practices, i.e., to identify a set of the techniques and algorithms which are appropriate for capturing deriving, or using PLUM models; finally, 3) to compare PLUM models with other more common approaches such as collaborative filtering. In addition, we hope to focus on the following problems and application domains, which are unique to PLUM-modeling:
The examination of ways to capture personal interaction histories to serve as personal memory prostheses has led us to the closely related field the automatic, continuous archival of personal experiences (i.e., life logging). Early work in this space was done by using mobile/wearable devices was done by Lamming et al. , a system which built an "automatic diary" based upon a user's physical movement in the workplace; and Rhodes et al. , which demonstrated how incorporation of sensed location and other physical context (such as the identity of nearby persons) could be used to retrieve notes created in similar contexts. Microsoft Research's more recent MyLifeBits project has sought to build a lifetime personal information store that correlates spatial temporal correlations among resources for retrieval tasks, such as searching for documents based on co-occurring events, co-located items, and time of access .
With respect to examining users' information manipulation behaviors at the desktop, the CALO IRIS project  has produced an instrumented desktop environment capable of recording every action the user performs. Additionally, IRIS uses a rich semantic-grounding for observations, which served as inspiration for our KR. PLUM differs from IRIS in that it attempts to instrument the user's desktop applications without modifying them noticeably to the user - which has a considerable adoption (and long-term maintenance) advantage over re-implementing an entire suite of desktop applications, and forcing users to switch to a foreign interface.
On the modeling frontier, a new open-source toolkit by Fogarty and Hudson called SUBTLE  seeks to simplify the process of building statistical models of users' activities. We plan to investigate SUBTLE's capabilities and techniques, and may consider integrating it with PLUM when the system is released.
With respect to context capture, we have demonstrated a working prototype of Chron, PLUM's capture framework, written in Java, with a set of knowledge sources designed for the Apple MacBook.  The current set of knowledge sources monitor a user's interactions with their desktop applications on Mac OS X (via integration with Applescript), filesystem activity, the user's location via WiFi, accelerometer readings from the Sudden Motion Sensor, and takes periodic images of the user using the integrated iSight. Observations are represented in RDF and written via the Jena RDF API. We have demonstrated that Chron works effectively with under 5 percent CPU utilisation (combined with mysql) when set to 3Hz with 10 knowledge sources. (On the same machine, iTunes consistently consumes 6-12 percent CPU while playing mp3s).
With respect to our second goal, of demonstrating the value of captured activity logs, we are currently prototyping an augmented personal journal application known as jourKnow, which establishes correspondences between when each piece of text in the journal was created to the user's environment and activities surrounding that moment. We are hoping to evaluate whether such correspondences can be used to facilitate re-finding within the journal, reduce the effect of memory decay on identifying the meaning of journal entries, and help users re-fill missing bits of their journal. See  for a preliminary discussion.
For more information
Please see PLUM's web site for code releases and project updates.
PLUM is part of the ConnectingME project, and is supported by CSAIL and the MIT-Nokia collaboration.
 Adam Cheyer, Jack Park and Richard Giuli. "IRIS: Integrate. Relate. Infer. Share." Workshop on the Semantic Desktop. Proceedings of the International Semantic Web Conference, Galway, Ireland, 2005.
 Max Van Kleek and Howard E. Shrobe. "A Practical Activity Capture Framework for Personal, Lifetime User Modeling". In Proceedings of 11th International Conference on User Modeling (UM 2007), Corfu, Greece, June 2007.