Tag sensing

Lively Revisiting Home Monitoring

One of the reasons why activity recognition in the home is interesting to me is because it has the potential to enable so many important health applications, such as remote monitoring. There are many people, especially older adults, who would much rather stay at home while battling chronic diseases than move into an assisted living facility or hospital. But without a range of supportive services, which are often prohibitively expensive, it becomes virtually impossible to properly care for someone in their own home.

There is a wide range of commercial products centered on supporting independent living, from fall detection to medication compliance systems. One piece of the puzzle that is missing is a communication channel that offers caregivers a holistic view of an individual’s patterns of daily living at home. This would enable caregivers to observe everyday behaviors on a regular basis and hopefully anticipate problems.

Today I read about the Lively system, a sensor network and base station designed for eldercare remote monitoring. It combines a number of wireless sensors that could be attached to objects:

One type of sensor goes on pill boxes, while another measures whether people are eating and drinking on a regular schedule by indicating when refrigerator or pantry doors are opened, both using accelerometers. A third variety is a key fob with a Bluetooth Low Energy transmitter than lets the server know when the user is out of range, typically 125 meters (about 410 feet). This measure acts as a proxy for indicating when the person has left home.

Researchers have attempted to use sensor networks this way, with moderate success. For example, Tapia took the idea of sensors in the environment and showed how one could learn more about an individual’s high-level activities from low-level sensors. Rantz and Skubic demonstrated how a sensor network could be used as an early-warning system for conditions such as urinary tract infection in older adults. The only limitation of these system has been the large number of sensors required, sometimes in the order of 50-80 sensors per home. That is too many.

It’s clear that there is room for sensor networks in health monitoring at home, and Lively is betting that a few strategically located wireless sensors can tell us most of what we need to know about someone’s well-being. I do agree with this direction, and I am now curious to see how the company does in the future, including whether it raises significant funds through Kickstarter, which is a good indicator of how demand exists for a product like this.

The Privacy of Smartphone Health Sensing

It seems like every other day a new story in the media reports on the rise of the smartphone as the uber-sensor for medical applications. Yesterday I had the chance to read the New York Times story “Apps Alert the Doctor When Trouble Looms“, which highlights apps that monitor patterns in device usage (e.g. phone calls made, text messages received) and link pattern deviations to a variety of possible medical conditions.

The notion of extracting behavior features from smartphone usage is an idea I am fond of. I am following Ginger.io, the company that brought this idea to market, very closely. So, naturally, I read the story with interest. Surprisingly, the most remarkable aspect of the story was not the story itself, but the comments from readers.

As of now, 8 out of 10 comments note that the notion of using a smartphone as a diagnostic tool or to alert a doctor is preposterous. The majority of these negative comments were “upvoted” using the “Recommend” link that the New York Times publishing platform provides. Many comments touched on the privacy issue:

This is insane Big Brotherism

No way around it, such is a Violation of Privacy!

An interesting perspective came from someone battling chronic conditions:

As someone dealing with several chronic conditions I feel qualified to state that this is the worst idea I have ever read. The possibilities for the app to go wrong are endless.

A doctor also chimed in:

You want to know the first thing i thought about as a doctor? Picturing me getting sued by a patient or their family and the lawyer saying: this app shows that you received this information, yet there is no record that you acted on it. im literally supposed to act on it, pull the chart, and then note what the app said and what i did, and why i did it, and why i didnt do something else. tort reform before i use this app.

Two observations.

First of all, the variety of viewpoints on an important topic from many of the represented parties constitute, in my opinion, one of the best practical examples of Social Construction of Technology theory (SCOT) at play I have ever witnessed. In a few words, human action shapes technology and technology is interpreted differently by different social groups.

Secondly, as we move forward developing technologies that assess an individual’s health condition from everyday behavior, do we need to pay more attention to these loud voices asking us to stop? Or should we remember that just 10 years ago, the idea of sharing what we share today on social networks was equally ludicrous and ended up going mainstream anyway? As a technologist, I am not surprised to find myself leaning towards the latter. But I do believe it’s time for more effort to be put into thinking deeply about the privacy implications of this work.

What do you think?

ISTC-PC 2012 Retreat

Due to a very hectic schedule in the last few months, which included a paper deadline, the Ubicomp conference, a qualifying exam presentation and the beginning of classes, I haven’t been able to blog here nearly as often as I should. With a little bit more breathing room now, I will try to catch up in the next post or two.

Let me start with the ISTC retreat. Back in August, I had the opportunity to attend the first ISTC-PC retreat in the beautiful Alderbrook Resort & Spa, located about 2 hours from Seattle by car and ferry. It was a great event, and an opportunity to meet up with other researchers that comprise the relatively new Intel Science and Technology Center for Pervasive Computing.

Many professors and researchers from the University of Washington were in attendance and it was wonderful to see old friends and meet new ones. Intel was well represented and sent a large entourage of engineers and executives from many relevant units, from embedded systems to mobile devices and personal computers. I found it particularly rewarding to meet and have conversations with Deborah Estrin (Cornell-NY), Tanzeem Choudhury (Cornell) and their students, since we all share a deep interest in health modeling and applications.

It was encouraging to hear that Intel is very eager to enable our vision of the future as well. In particular, one of the issues they are addressing is how to make mobile device sensors less dependent on the CPU and associated chip infrastructure in the interest of conserving battery life. Designing so-called “sensor hubs” might be one direction to pursue.

Overall it was a great experience and I am looking forward to more interactions with ISTC collaborators and Intel in the future.

Recognizing Home Activities with IMS

Today I am in Pittsburgh for Ubicomp 2012, presenting our work in recognizing activities in the home using infrastructure-mediated sensing:

Recognizing Water-Based Activities in the Home Through Infrastructure-Mediated Sensing

Abstract
Activity recognition in the home has been long recognized as the foundation for many desirable applications in fields such as home automation, sustainability, and healthcare. How- ever, building a practical home activity monitoring system remains a challenge. Striking a balance between cost, pri- vacy, ease of installation and scalability continues to be an elusive goal. In this paper, we explore infrastructure-mediated sensing combined with a vector space model learning ap- proach as the basis of an activity recognition system for the home. We examine the performance of our single-sensor water-based system in recognizing eleven high-level activi- ties in the kitchen and bathroom, such as cooking and shav- ing. Results from two studies show that our system can es- timate activities with overall accuracy of 82.69% for one in- dividual and 70.11% for a group of 23 participants. As far as we know, our work is the first to employ infrastructure- mediated sensing for inferring high-level human activities in a home setting.

Authors
Edison Thomaz, Vinay Bettadapura, Gabriel Reyes, Megha Sandesh, Grant Schindler, Thomas Plo ̈tz, Gregory D. Abowd, Irfan Essa

Download the full paper and leave your questions, comments and suggestions here.

Health Sensors Approaching Escape Velocity

Rock Health, a technology/startup incubator focused on health companies and applications, just released a very clear report on medical and wellness sensors. It’s a graphical, somewhat summarized version of a post I wrote a while back about health sensing technologies. Definitely worth a look:



The key message here is that health-focused sensors will be big. They project hundreds of millions of devices by 2014.

The $25 Raspberry Pi

Lately, as part of my research in indirect health inference through infrastructure-mediated sensing techniques, I’ve been investigating options for remote data acquisition. It would be wonderful if I could find a platform that let me take an analog signal as input, send it through a pipeline of signal processing and machine learning algorithms, and submit results to a remote server.

At the high-end, there are netbooks. Any BestBuy can sell you a complete netbook system for less than $300. For certain applications in data sensing, processing and communication, $300 is good enough. Unfortunately, a netbook is a bit too big and power hungry. Not to mention that its screen, graphics card and other features might go unused, inflating the cost of the device unnecessarily considering the job it’s been designated to do.

At the other end of the spectrum are platforms like the Arduino, which is small, inexpensive, but might not provide the processing power one might need. Are there any other alternatives? There are plenty of single-board computers out there, one of which is the Chumby Hacker Board, or CHB for short.

Recently, I’ve been following the development of an ARM-based platform called Raspberry Pi. The goal is to develop the cheapest possible computer with a basic level of functionality, for around $25. The team is already showing a prototype board, the size of a credit card, running Ubuntu:

ubuntu.jpeg


Lots of details can be found here and they are expected to be shipping in December. I am really looking forward to experimenting with them.

Hydrosense: The U.S. and Beyond

The primary sensing technology that I’ve been using in my health modeling research this year is Hydrosense, a device that consists of a water pressure sensor and some additional hardware for signal acquisition, processing, communication and/or storage. By monitoring water pressure change patterns in a single-family home, Hydrosense lets us identify which water fixtures are in use and help us develop a good sense of which water-based activities are taking place. It’s a cornerstone of our activity and lifestyle recognition efforts.

This technology was conceived and originally designed by Shwetak Patel, Joe Froehlich, Eric Larson and others at the University of Washington.

A few weeks ago, while at the CDC Public Health Informatics Conference, a question was posed to me: “What is the Hydrosense coverage right now considering that not all homes in the US are single-family homes and not all of them are connected to a water supply system?”. I was intrigued by this question and decided to investigate further. Luckily I didn’t have to go very far. Wikipedia and the U.S. Census Bureau had all the numbers I was looking for.

There are 115.9 million homes in the US. About 70 million of these (60.3%) are detached single-family units. Eighty percent of single-family homes are occupied by owners. In terms of water supply, 14.5% of Americans rely on their own water sources, typically wells. Water well pumps are used in this case. The pump sends water to a storage tank with an air bladder that compresses as the water is pumped in. At 40-60psi, the pump stops. When water is used in the home, pressure drops and when it goes below 20psi, the pump starts again.

More than 99% of the US has access to “complete plumbing facilities”, defined as having (1) hot and cold piped water, (2) bathtub or shower, and (3) flush toilet. Homes that lack such water facilities total 670,986, and are usually inhabited by the elderly, the poor and those living in rural areas. Alaska has the highest percentage of households without plumbing.

To sum it up, Hydrosense can be used today in 60% of homes in the US. We would like to enhance it so that it can work reliably in multi-family homes and apartment complexes as well, which will expand Hydrosense’s coverage to virtually every home in the country. Thinking globally, and especially in the context of developing countries, I now wonder how suitable Hydrosense is in other regions of the world.

Biomechanical Analytics

orthosensor.png

The field of orthopedics is starting to reap the benefits of ubiquitous computing. While over at MedGadget, I found out about OrthoSensor, a company that is developing a platform around embedded sensing for joint replacement implants. Among other benefits, this system will make it possible to wirelessly monitor the bio-mechanical performance of a knee replacement over the long term, for example.

Together with capsule endoscopy and digestible sensors, embedded bio-mechanical analysis represents another step towards the future and mainstream reality of bioelectronics.

Health Sensing: Present and Future

Human health sensing plays a key role in my research. Over the last few years there’s been a sharp incline in the quantity and variety of consumer devices and medical sensors that capture some aspect of physiological, cognitive and physical human health.

This post is my attempt to capture some of the activity in this space. This is not an exhaustive list, but I hope it’s representative of where we are. It combines devices and sensors that are fairly mainstream with hundreds of thousands of users, with others that are still in their infancy as research projects. I might update this list sporadically, based on new findings. I am also curious to find out what is missing from the list, so feel free to leave comments.

Activity Tracking

BodyMedia: The BodyMedia FIT system provides activity, calories and sleep pattern data. The BodyMedia FIT Armband automatically tracks the calories burned during daily activities and monitors the quality of your sleep. This is a fairly popular device and has been around for quite some time. As the name suggests, it is attached to the body through an armband and collects movement data.

Basis: Basis is a wristwatch that tracks caloric burn, activity levels and sleep habits. It’s not available yet, but it looks like it will be soon. The form factor is promising, and looks good too. It is supposed to contain an optical blood flow sensor, obtain temperature readings, and more.

Fitbit: The Fitbit tracks calories burned, steps taken, distance traveled and sleep quality. It’s a small device that can be clipped to a belt. It’s also quite popular, and available for purchase today.

Jawbone Up: New upcoming product announced by Jawbone, the company behind highly-rated headsets. It will track movement, sleep patterns and eating habits and will have the form factor of a bracelet.

Valencell Healthset: Earbuds that track heart rate, calories burned and physical activity. If this really works as advertised, it has enormous potential, since it doesn’t require any additional sensing devices like the BodyMedia FIT and the Fitbit.

Zephyr: The key product from Zephyr is called BioHarness, which measures critical vital signs (ECG, heart rate, breathing rate, skin temperature ) and physical activity using an accelerometer. The activity and physiological data can be transmitted in real-time for remote condition monitoring. They don’t seem to be selling a consumer product right now.

24eight: This company is developing a whole range of technologies associated with health sensing, from “smart” slippers (with “smart” insoles – slippers that can tell when your grandmother might be headed for a fall) to SIDSense, a 3D infant mobility monitor sensor. From what I can tell, 24eight is developing technologies and licensing it to be used in various products.

Sleep

Zeo: Zeo is the leader in the space of sleep analytics. After purchasing a $200 kit, you wear a headband when you go to bed. In the morning, the Zeo scores the quality of your sleep and shows you detailed analytics of how you slept through the night. A feature I find compelling is the alarm clock, which wakes you up within a time window, whenever it senses you will be most refreshed.

WakeMate: Similar to the Zeo, but instead of a headband, you must wear a bracelet. They claim their analytics platform “optimizes your waking hours by automatically analyzing your sleep and illuminating personal habits that affect your sleep”.

Lark: Similar to the Zeo and the WakeMate. Lark also markets the product as a soundless alarm clock – the bracelet vibrates when it’s time to wake up, without disturbing the significant other sleeping next to you.

Nyx Devices: Nyx is developing the Somnus Sleep Shirt, a tshirt with embedded respiration sensors. This is another approach for obtaining sleep analytics data and a significant step in the direction of wearable sensing. I am really looking forward to seeing the future of this product.

Blood Glucose Level

Dexcom Seven Plus Continuous Glucose Monitoring: This device is a continuous glucose monitoring system for people with diabetes. Unlike traditional glucose monitors that require a blood sample for each analysis, patients introduce a very small sensor/transmitter into their bodies that sends glucose level information to a receiver outside the body in real-time. The sensor is placed subcutaneously and continuously measures glucose levels in the interstitial fluid.

iBGStar Blood Glucose Meter: The iBGStar is the first available blood glucose meter that connects to the Apple iPhone. I fully expect to see more devices connecting and sending data to mobile devices, so this device is emblematic of this upcoming trend.

Remote Health Monitoring

GE QuietCare: This is a system for activity detection and recognition in the home. It’s the result of a partnership between Intel and GE. Data sent from the sensors is analyzed by algorithms to detect any out-of-the-ordinary events that may put residents at risk. We are just starting to see the category of health telematics and personal health monitoring emerge and this company is hoping to bring it to scale.

Others

Asthamapolis: A project whose goal is to track when an asthma inhaler is used. In connection with a mobile phone, a GPS-powered inhaler maps and track asthma symptoms/triggers, helping patients learn more about asthma while also improving public health. I anticipate an increasing number of medical devices associating its use with contextual information like time of day and location.

Affectiva Q-Sensor: Affectiva produces the Q Sensor, a wearable, wireless biosensor that measures emotional arousal via skin conductance, a form of electrodermal activity that grows higher during states such as excitement, attention or anxiety and lower during states such as boredom or relaxation. This is not a consumer product, but it’s rapidly becoming a popular tool in research studies where tracking emotional levels is required.

Duo Fertility Monitor: DuoFertility is a fertility monitor to help women get pregnant naturally, and avoid invasive medical procedures such as IVF. A patch-like sensor monitors body temperature and transmits it to a receiver, which then calculates a fertility level. As part of the service offered, fertility experts might also get involved to examine the data and provide feedback.

CellScope: An attachment that clips onto the back of an ordinary camera phone and turns it into a portable microscope capable of visualizing single-celled pathogens like malaria parasites or tuberculosis bacteria. This is a very powerful idea, especially in the context of developing nations.

Withings Scale and Blood Pressure: Withings is trying to make it easy to track basic physiological signals like blood pressure, BMI and weight over time. They have developed devices that transmit data wirelessly over WiFi or to a mobile phone with the goal of making it easy to get a daily health overview.

Medical/Clinical

Proteus Biomedical: Proteus develops ingestible event markers (IEMs), tiny, digestible sensors made from food ingredients, which are activated by stomach fluids after swallowing. Once activated, the IEM creates an ultra-low-power, private, digital signal detected by a microelectronic recorder configured as either a small bandage style skin-patch or a tiny device inserted under the skin. The detector date- and time-stamps, decodes, and records information such as type of drug, dose, and place of manufacture, and also measures and reports physiologic parameters such as heart rate, activity, and respiratory rate. Detector data can be combined at the server-level with other telemetered parameters such as blood pressure, weight, blood glucose, and patient-generated feedback. Quite remarkable.

Given Imagine: This company designed the PillCam video capsule for capsule endoscopy, a medical procedure which allows your physician to visualize parts of your gastrointestinal tract. The GI tract is a part of the digestive system and extends from the mouth to the anus. This is very impressive technology.

There’s no doubt we will see an explosion of human sensing devices over the next 5-10 years. What is available today is just scratching the surface of what is possible. Two major trends are already evident, (1) the mobile phone will become the hub to many of these sensors and devices (although I expect that we will see increasingly more sensors that are capable of transmitting data wirelessly over cell networks) and (2) we will see a lot of experimentation with non-invasive body sensing (e.g. tracking sodium and glucose using nano-sensors with a bio tattoo and a smart phone).

For additional resources related to this topic, I suggest the Quantified Self Guide to Self-Tracking. The guide lists applications and services beyond health and medicine sensing, but it contains lots of examples of human health sensing tools.

Genetic Testing and (Maybe) Behavior Analysis Against Alzheimer’s

Alzheimer’s is a tough disease. It’s incurable, and there no treatments to delay or halt its progression. In the Journal of Neuroscience, Paul Thompson and colleagues at UCLA suggest that one of the risk-related genes begin to do damage to the brain 50 years before the disease is perceived.

If that’s true, why don’t we show signs of dementia. It just so happens that in youth, our brains are so rich in connectivity and redundancy that the problematic areas can be “bypassed” without major problems. But later on, with the compounded effect of aging, Alzheimer’s emerges in full force.

The sooner we can identify the presence of the disease, the more strategies we might have for reducing cognitive impairment. Genetic analysis is one direction, but can we use everyday behavior analysis to catch glimpses of the disease years before it’s clinically diagnosed? That’s one of the hypothesis underlying my research work.

More details here.