Summer is coming to an end quickly, as it always happens. In fact, at Georgia Tech, Fall classes started this week. Most of my time in July and August was dedicated to examining on-body sensor streams and evaluating ways to process and make sense of them. I am getting ready to run some studies in the Fall, and it was immensely useful to concentrate on improving the methods and techniques I will be relying on for analysis and activity inference.
At the end of July, I had the chance to visit Prof. Bonnie Spring’s lab in the School of Preventive Medicine at Northwestern University. We are exploring a collaboration where some of the eating recognition systems that I am building as part of my dissertation work will be employed in diet/behavior studies. I will save all the details of this possible collaboration to another post, but I am thrilled about the opportunity to work with Prof. Spring and her team. In my view, working closely with experts in the medical sciences beyond computer scientists interested in addressing health problems is critical to making meaningful impact. Hopefully I will have more to share soon.
Finally, just last week I attended the third annual Intel Science and Technology Center for Pervasive Computing (ISTC-PC) retreat, which was held close to Seattle, WA. Intel has been a major support of my graduate research work. As expected, I met many colleagues and had numerous productive conversations. Joe Paradiso gave the keynote speech after dinner:
Intel executives and researchers often come to this event. Bruce Horn, one of the members of the original Macintosh team, and none other than the man who created the Finder, is now the head of an Intel research unit. His signature is inside the case of the original Mac 128K, alongside other legends like Jobs, Raskin, Espinosa and Hertzfeld:
Here is a great post where he describes joining the Mac team. Naturally, I had to ask him to sign my MacBook Air:
Fun times. I hope the Georgia Tech folks tolerate my appreciation for the history of computing.
Really? Well, at least that’s what the authors of this paper published at the Workshop on Physical Analytics (Mobisys 2014) claim. Yes, I did say I was focusing on CHI2014 papers over the next few weeks, but I ran into this one and had to read it. After all, my dissertation topic is in large part about inferring eating activities.
Authors: Chen, Wang, Zhou, Campbell (Dartmouth College)
Idea: Use of behavior data and location history from smartphones to predict if an individual is going to eat or not in the future. The goal is to provide just-in-time feedback to people about healthy food and drink choices.
Approach: Built a CART (classification and regression trees) model to predict food purchases of a group of undergraduate students. Top six features were (1) the current building, (2) arrival time of current building, (3) departure time of previous building, (4) activity ratio of last building, (5) arrival time of previous building, and (6) conversation duration at previous building. A prediction baseline was set with random guessing and compared with a generic model (10 fold cross-validation on all the data), personalized model with and without adaptation. With adaption, the prediction model is trained for each individual and updated over time to reflect most recent behavior.
Study: 10-weeks, 25 undergraduate students, used dining card purchase records as ground truth.
Results: 74.2% accuracy, 52.7% precision, 55.1% recall (using three weeks of training, personalized model with adaptation).
Published: Workshop on Physical Analytics (WPA) 2014 – Mobisys 2014
As part of my CHI 2014 paper reading series, today I read “Personal Tracking as Lived Informatics” by Rooksby et al. from the University of Glasgow in the UK. They offer a compelling take on activity tracking. It is a good read, even if a bit contradictory at times, in my opinion.
Idea: Understand people’s practices and relationships with activity tracking devices and personal tracking in everyday life. This paper claims that personal tracking has been studied very narrowly so far, where devices serve as nothing more than activity detectors or as part of interventions.
Approach: Unstructured and follow up interviews with 22 participants.
Findings: Tracking is social (but not in the ’share on Twitter/Facebook’ social, there are different styles of tracking (directive, documentary, diagnostic, reward-based, gadget-based), multiple trackers are used, tracking information used for meeting daily or short-term goals.
Notes: People chose different metrics to examine out of the devices they used, data collected and analyzed for short-term use (people not concerned about saving the data), tracking punctuated memorable life experiences (e.g., finishing a marathon), self-esteem. Idea that tracking is actually almost always prospective, since it’s about where one’s heading (e.g., losing weight, training for a marathon.
Over the next couple of weeks I will be writing several posts about CHI 2014 papers that caught my eye. I will format my summaries in 4 short paragraphs: Idea, Approach, Study, Notes. This structure will fit most papers I think. I will start with a paper by Matthew Lee and Anind Dey titled “Real-Time Feedback for Improving Medication Taking“. As I understand it, this is Matt’s PhD work at CMU under Anind. Matt is now at Phillips Research.
Idea: Study the effect of an instrumented pillbox and ambient display providing real-time feedback on medication taking. Research questions: (1) does real-time feedback improve consistency of medication taking?, (2) does real-time feedback improve self-efficacy for medication taking?, and (3) do medication taking behaviors change when removing real-time feedback after long-tern use?
Approach: Added various sensors and communication capabilities to a pill-box. Activity uploaded to server, projected back onto a visualization on a tablet that participants kept around the house.
Study: Longitudinal study with 12 people over 10 months. Participants divided in control, feedback groups. Two months for baseline, no feedback for both groups. After 6 months, real-time feedback was removed from feedback group.
Notes: The real-time feedback improved medication-taking behaviors and self-efficacy, but when it was removed after long-term use, performance was not sustained.
Together with Jon Froehlich, Jakob Eg Larsen and Matthew Kay, I am organizing a Ubicomp workshop later this year. The theme: disasters in personal informatics. Here is a synopsis of what we are trying to accomplish:
In this workshop, our goal is to uncover, analyze, discuss, and learn from the failures of PI and QS research—failures that are most often not captured or surfaced in traditional publications because of embarrassment, perceived irrelevance, or simply lack of space. We want to provide an explicit forum to share stories of failure, perhaps even entire lines of research that did not succeed, in order to synthesize lessons learned and help progress the PI research community forward.
More details can be found on the official workshop web site. Please considering submitting – I believe this will be a rewarding experience for all who are working in the Personal Informatics space.
Unlike many of my colleagues, I am not going to CHI this year. Instead, I am staying put in Atlanta and laying the groundwork for some of my upcoming studies. With my many April deadlines and to-dos behind me, I am looking forward to resuming my paper reading routine. This morning I set aside 20 minutes for a paper published all the way back in 1939, “Long Term Study of the Variation of Serum Cholesterol in Man“.
In this work, Turned and Steiner examined how cholesterol levels change in individuals over time. They had the opportunity to study 10 hospitalized patients over a period of 12-14 months at approximate weekly intervals. These patients were tested under different conditions (e.g., administration of thyroid medication, on a high-fat diet, on a low-fast diet, etc.). Thyroid admin resulted in drop in cholesterol and rise in basal metabolic rate, whereas a high fat diet caused a small increase in cholesterol. A low fat diet failed to influence cholesterol level.
Last week I had the opportunity to attend the Quantified-Self Public Health Symposium organized at UCSD by Bryan Sivak (U.S. Department of Health and Human Services), Larry Smarr (Calit2 and ), and Gary Wolf (Quantified Self Labs). The intent of the one-day event was to provide a conducive environment for researchers, policy makers and developers of QS-type tools to discuss issues around data and its potential for use in public health. Many attendees such as Susannah Fox have written about the highlights of the day from their perspective. Here are my notes:
At the beginning of the day, Larry Smarr said he believes this is the “year of scale” when it comes to personal health informatics. Stephen Downs (Robert Wood Foundation) reiterated this point with a remark about our “rapidly expanding oceans of personal health data” and expanded on how RWJ Foundation has identified that a cultural change is needed around the value of health in society. Gary Wolf pointed out the uniqueness of the researchers involved in QS, ranging from public health epidemiologists and CS/HCI types, and also claimed that we are at “the beginning of a new program in the human sciences”, which I very much agree. Susannah Fox from the Pew Research Center made a number of excellent points based on the data she has been collecting and studying. When it comes to self-quantification, do “people want the unforgiving bright light of numbers in front of them”?, she asked. Fox also talked about how it is important to respect cultural rituals.
Later in the day, in a session reserved for “toolmakers”, Anne Wright from Fluxtream discussed how personal health data should be as portable as personal financial data, where one can download documents from banks and upload them to tools such as Mint and Quicken. In terms of visualizing and making sense of data, she mentioned the need for different types of interfaces at different levels (e.g., personal dashboards versus deep history introspection). Margareth McKenna from Runkeeper brought with her a number of research questions that are difficult to answer even in light of large amounts of data, such as what happens when people are not logging their exercise and activities (“sensors can only capture so much”). She also emphasized the need to disseminate tools to everyone, echoing others during the day who talked about the “digital health divide”. Andy Hickl from A.R.O. presented the “11-week problem”, at which point self-tracking devices go to the drawer. In other words, “what should we care?”, or how can we make self-tracking data meaningful? At the end of the session, Gary Wolf came back and proposed a research toolkit that could be shared within the community.
In one of the sessions focused on study designs, a highlight for me was Eric Hekler, a behavior psychologist and deep thinker around behavior change. He discussed how he lost faith in randomized clinical trials and how we should be asking people what they truly want from data and devices. He also made the point that we should support people to come up with their own interventions and find out ways to “get more functional life years”.
To me, the high-point of the day was a panel with Larry Smarr and Lee Hood. Hood, whose work has revolutionized biology and genetics, is leading a longitudinal, Framingham-like study with the goal of better understanding how digital technologies and ongoing self-tracking can be leveraged to quantify what it means to be healthy. Another goal of the study is to look at the progression of disease, as observed by these devices. Smarr, one of the most recognized self-trackers in the Quantified-Self movement today, started his tracking journey to understand his own personal health issues, in a way doctors could not, or were simply unwilling to listen.
In addition to the plenary talks and discussions, the day was filled with coffee breaks, where I had the chance to chat and exchange ideas with many researchers and developers, such as Kevin Patrick from UCSD and Mike Lee, who leads MyFitnessPal. Overall, a great day in the campus of UCSD.
A week ago today, I finally managed to get my committee in one room and presented my PhD proposal to them. The title of my thesis is “Interactive Activity Recognition for Practical Food Journaling”.
The proposal is very important because it is essentially a list of work items that I need to complete before I can defend my thesis. I am now done with this milestone and have a roadmap for the next several months in hand. Without a doubt, I will be writing more here about interactive activity recognition. Onward!
A new year is upon us, hello 2014. This is the time of the year when I look back and wonder how 12 months went by so quickly. I am sure I am not alone. The last couple of months of 2013 were particularly busy. In addition to personal trips during the holidays, I also had the opportunity to attend the 2013 SenseCam and Pervasive Imaging Conference in San Diego, CA.
This was a small conference focused primarily on applications of first-person point-of-view images. I presented our paper titled “Feasibility of Identifying Eating Moments from First-Person Images Leveraging Human Computation”, which can be downloaded here. Our work was very well received and I had the opportunity to meet several researchers at UCSD and beyond who are taking health and behavior assessment to a whole new level thanks to wearable cameras.
Next on the agenda are additional studies related to my thesis work and my thesis proposal, which will probably take place at the end of January. As @dgmacarthur wrote on Twitter:
“May 2014 bring you clean data, well-documented code, large and deeply phenotyped samples, and a clear path to clinical translation”
While getting ready to start my day this morning, I flipped through the latest edition of the “Communications of the ACM” magazine. Right at the beginning was a very compelling writeup by Jason Hong about privacy in the age of Google Glass and similar wearable devices. Jason makes two points. In the first one, he draws a parallel between what we are experiencing today with regards to privacy in terms of wearable devices and what emerged when the “Invisible Computing” vision of ubiquitous computing was first put forth by Mark Weiser. Back in the late 80s and early 90s, there was also an outcry with regards to the negative impact of technology, with the press writing stories with titles such as “You’re Not Paranoid: They Really Are Watching You”.
According to Jason, it all comes down to expectations, and more importantly, the fact that we don’t know what to expect from these technologies, and whether they will be useful enough to offset the privacy concerns. The second point in Jason’s argument is that expectations of privacy change. He mentions Claude Fisher’s work observing that at first people objected to having landlines in their own homes and, of course, brings up the stir caused by the introduction of the Kodak camera when all of a sudden, any moment could be captured in film.
A good short read, and it’s also available online.