Introduction

Many people use day planners as scheduling aids. However, no one formally schedules every minute of their day. So how do people spend those parts of their day that aren't explicitly accounted for in a day planner? We are interested in using a person's typical activity patterns, the unscheduled items on his to-do list, and his personal preferences to predict the places he visits during his unscheduled time, and to make relevant suggestions regarding how to spend this time, given his current location and schedule.

Scenarios

Three different scenarios illustrate the sorts of inference we would like to be able to do.

• Opportunistic side-trips. June is a student at Wellesley College. Among other things on her to-do list, she has to pick up some items she had dry-cleaned, she has to buy party decorations, and she has to return a shirt to a department store. Today, June has a 3pm class at MIT. Since the store at which she needs to make her return has no outlets near Wellesley, June decides to take the shirt that needs to be returned and leaves Wellesley at 1pm. This allows her with enough time to visit the mall near MIT and return the shirt before her class. Without knowing that she took the shirt with her, is it possible to reason that June left Wellesley early to return her shirt?
• Bus, subway, or drugstore? Gary is in Central Square. He normally takes the bus home from Central, but he could also take the subway home and then go shopping at the grocery store near his subway stop. Gary also has "buy shaving cream" on his to-do list, which he can do at the drugstore near the Central bus stop. Under what circumstances will Gary get on the bus, the subway, or go into the drugstore?
• Wandering and dilly-dallying, or wanting to dine? Gary has a 6pm dentist appointment at Kenmore Square. He decides to leave for his appointment at 4pm, allowing him ample time to explore the shops in Kenmore Square. The next week, Gary has tickets to the 6pm Red Sox game. He again leaves work early at 4pm and heads to Kenmore Square, but this time, he leaves early so that he has enough time to eat dinner before the baseball game. We would like to be able to distinguish between these two situations, and provide Gary with appropriate suggestions in each and perhaps reminders to buy shaving cream in the first but not in the second.

Approach

To make the sorts of inferences described in the above scenarios, we break the problem into the following parts. First, we need a descriptive location representation that can be used to describe not only where a person is, but also what he can do at nearby places. We also need to be able to learn a person's usual activity patterns and reason about where a person would go to carry out to-dos, if this information is not included in the to-do. Finally, if, as in the case of the second scenario, there are multiple places a person is likely to go next, we need to be able to rank the likelihood that he would go to each place.

We address these sub-problems as follows. We have created LAIR, an ontology that models both the geographical relationships between spaces as well as what can be done at a given space [2]. We intend to use work by Don Patterson [3] to learn a person's typical activity patterns and the common-sense knowledge in ConceptNet [1] to infer where different items on a to-do list can be done.

Once we have identified a person's usual activity patterns and the places at which he can complete his to-dos, we want to generate a set of rules that reflect both his preferences (as demonstrated by which places he has visited in the past) and certain decision-making heuristics. One such rule could state that if a person has an appointment far from where she spends most of her time, then she could use that appointment as an opportunity to visit other places that she would not otherwise be near. Such a rule could be used in Scenario 1 to infer that June was headed to the mall before her 3pm class.

Another rule could use the frequency at which a person visits a certain place to predict whether or not he will visit that place in the near future. Such a rule could be used in Scenario 2 to predict whether Gary will get on the subway and then go grocery shopping or whether he will catch the bus or buy shaving cream at the nearby drug store. A third rule could be used to distinguish between situations in which a person is willing to entertain a suggestion to visit a certain place and perform a to-do, versus disregarding the suggestion because it was made at an inconvenient time (such as a suggestion to buy shaving cream in Scenario 3).

While we initially plan to hand generate these rules, we would like to be able to eventually learn these rules by using past behavior as input to machine learning algorithms and by incorporating explicit user feedback.

Research Support

This research is supported by MIT's Project Oxygen.

References

[1] Hugo Liu and Push Singh. (2004) ConceptNet: A Practical Commonsense Reasoning Toolkit. In BT Technology Journal, To Appear, Volume 22, forthcoming issue.

[2] Gary Look, Buddhika Kottahachchi, Robert Laddaga, and Howard Shrobe. A Location Representation for Generating Descriptive Walking Directions. In IUI '05: Proceedings of the 10th International Conference on Intelligent User Interfaces, pp. 122-129, San Diego, CA, USA, January 2005.

[3] Donald J. Patterson, Lin Liao, Krzysztof Gajos, Michael Collier, Nik Livic, Katherine Olson, Shiaokai Wang, Dieter Fox, and Henry Kautz. Opportunity Knocks: a System to Provide Cognitive Assistance with Transportation Services. In The Proceedings of UBICOMP 2004: The Sixth International Conference on Ubiquitous Computing , pp. 433-450, Nottingham, England, October 2004.